[DEBUG] [06/30/2025 09:50:04.698] [main] [EventStream(pekko://test)] logger log1-TestEventListener started 09:50:04,698 AM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started [DEBUG] [06/30/2025 09:50:04.698] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started [DEBUG] [06/30/2025 09:50:04.699] [main] [EventStream(pekko://test)] Default Loggers started 09:50:04,714 AM [test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.12:17355] with UID [-7182390249696665640] [INFO] [06/30/2025 09:50:04.713] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.12:17355] with UID [-7182390249696665640] [INFO] [06/30/2025 09:50:04.714] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - Starting up, Pekko version [1.0.3] ... 09:50:04,714 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - Starting up, Pekko version [1.0.3] ... [INFO] [06/30/2025 09:50:04.716] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 09:50:04,716 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - Registered cluster JMX MBean [pekko:type=Cluster] [INFO] [06/30/2025 09:50:04.716] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - Started up successfully 09:50:04,716 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - Started up successfully [INFO] [06/30/2025 09:50:04.717] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing [INFO] [06/30/2025 09:50:04.718] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:50:04,718 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:50:04,718 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:50:04,727 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries starting: sending 2 new payloads [WARN] [06/30/2025 09:50:04.731] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower760]: org.opendaylight.controller.cluster.common.actor.Monitor@2edc0995 [INFO] [pekkoDeadLetter][06/30/2025 09:50:04.731] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower760] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:04,731 AM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower760] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:04,734 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - follower760: Starting recovery with journal batch size 1000 09:50:04,734 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower760: started executor 09:50:04,734 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower760: started executor 09:50:04,735 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower760: metadata journal open: replayFrom=1 applyTo=0 09:50:04,736 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower760: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:04,736 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower760: journal open: applyTo=0 09:50:04,736 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: journal writer started 09:50:04,736 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:50:04,737 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/follower760/TermInfo.properties does not exist 09:50:04,737 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/follower760/journal-v1-1.log 09:50:04,737 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/follower760/metadata-v1-1.log 09:50:04,737 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower760: no eligible files found 09:50:04,737 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower760: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:50:04.738] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower761]: org.opendaylight.controller.cluster.common.actor.Monitor@7988665c [INFO] [pekkoDeadLetter][06/30/2025 09:50:04.737] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:04,738 AM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:04,738 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - follower761: Starting recovery with journal batch size 1000 09:50:04,738 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: started executor 09:50:04,738 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: started executor 09:50:04,738 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower761: metadata journal open: replayFrom=1 applyTo=0 09:50:04,739 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower761: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:04,739 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower761: journal open: applyTo=0 09:50:04,739 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: journal writer started 09:50:04,739 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: waiting to receive commands [WARN] [06/30/2025 09:50:04.740] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader759]: org.opendaylight.controller.cluster.common.actor.Monitor@6bbd03e1 [INFO] [pekkoDeadLetter][06/30/2025 09:50:04.741] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader759] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:04,741 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - leader759: Starting recovery with journal batch size 1000 09:50:04,741 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: started executor 09:50:04,741 AM [test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader759] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:04,741 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: started executor 09:50:04,741 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/follower761/TermInfo.properties does not exist 09:50:04,741 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader759: metadata journal open: replayFrom=1 applyTo=0 09:50:04,741 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/follower761/journal-v1-1.log 09:50:04,741 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/follower761/metadata-v1-1.log 09:50:04,741 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower761: no eligible files found 09:50:04,741 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower761: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:04,741 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader759: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:04,741 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - leader759: journal open: applyTo=0 09:50:04,741 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: journal writer started 09:50:04,741 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting to receive commands 09:50:04,742 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/leader759/TermInfo.properties does not exist 09:50:04,743 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/leader759/journal-v1-1.log 09:50:04,743 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7084470151281463027/leader759/metadata-v1-1.log 09:50:04,743 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: no eligible files found 09:50:04,743 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=5, votedFor=leader759 09:50:04,743 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader759: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:04,743 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=0, votedFor=null 09:50:04,743 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower760: Local TermInfo store seeded with TermInfo{term=0} 09:50:04,743 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower760: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@c622999 09:50:04,743 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from -1 to -1 09:50:04,743 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower760: journal recovery completedin 211.5 μs with journalIndex=1 09:50:04,744 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=0, votedFor=null 09:50:04,744 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower761: Local TermInfo store seeded with TermInfo{term=0} 09:50:04,744 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower761: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@603923f9 09:50:04,744 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to -1 09:50:04,744 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower761: journal recovery completedin 343.3 μs with journalIndex=1 09:50:04,746 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=5, votedFor=leader759 09:50:04,746 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader759: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader759} 09:50:04,746 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader759: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@44502697 09:50:04,746 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from -1 to -1 09:50:04,746 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - leader759: journal recovery completedin 159.1 μs with journalIndex=1 09:50:05,213 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:50:05,213 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader759 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:05,213 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader759 (Follower): Received ElectionTimeout - switching to Candidate 09:50:05,213 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader759 (Candidate): Election: Candidate has following voting peers: [follower761, follower760] 09:50:05,217 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=6, votedFor=leader759 09:50:05,217 AM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader759 (Candidate): Starting new election term 6 09:50:05,217 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader759 (Candidate): Sending RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} to peer follower761 09:50:05,218 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader759 (Candidate): Sending RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} to peer follower760 09:50:05,218 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader759 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:50:05,218 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:05,218 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Cluster state: [] 09:50:05,218 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:05,218 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower761 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:05,218 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower760 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:05,219 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower760 (Follower): Cluster state: [] 09:50:05,219 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower760 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:05,219 AM [test-pekko.actor.default-dispatcher-7] [INFO] Follower - follower760 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:05,222 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=null 09:50:05,222 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): In requestVote: RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:05,223 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=6, votedFor=null 09:50:05,223 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower760 (Follower): In requestVote: RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:05,225 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=leader759 09:50:05,225 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:05,225 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader759 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:50:05,225 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader759 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:50:05,225 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Election: Leader has following peers: [follower761, follower760] 09:50:05,225 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:05,225 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:05,225 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader759 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:50:05,225 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower761: Last sync leader does not match current leader leader759, need to catch up to -1 09:50:05,226 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: -1, nextIndex: 0 09:50:05,226 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:05.226] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=false] 09:50:05,226 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=6, votedFor=leader759 09:50:05,226 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower760 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.226] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,226 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower760: Last sync leader does not match current leader leader759, need to catch up to -1 09:50:05,226 AM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,226 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: -1, nextIndex: 0 09:50:05,226 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:05.226] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower760] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower760, initialSyncDone=false] [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.227] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower760] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,227 AM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,244 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:50:05,244 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader759: Persist data index=0 term=6 command=zero 09:50:05,244 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader759: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:05,244 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:50:05,244 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:05,244 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:50:05,244 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 502.8 ms 09:50:05,244 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:05,245 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:05,245 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower760 09:50:05,245 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower760: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader759: Persist data index=1 term=6 command=one 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader759: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Number of entries to be appended = 1 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:05,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower760 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Append entry to log zero 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower760: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:05,246 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 510.2 ms 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Log size is now 1 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower760, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:05,246 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached -1 (of expected -1) 09:50:05,247 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower760: awaiting more completions to resolve 1 deferred callback(s) 09:50:05,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: 0, nextIndex: 1 09:50:05,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:05,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 1 is present for follower follower760 09:50:05,248 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower760: AppendEntries{term=6, leaderId=leader759, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:05,249 AM [leader759-writer-162] [DEBUG] EntryStoreCompleter - leader759: 1 completion(s) pending 09:50:05,249 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 5.098 ms 09:50:05,249 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 5.940 μs 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader759: running 1 completion(s) 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader759: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader759 (Leader): Setting last applied to 0 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from -1 to 0 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader759: Applying state for log index 0 data zero 09:50:05,250 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader759: applyState called: zero 09:50:05,251 AM [follower760-writer-160] [DEBUG] EntryStoreCompleter - follower760: 1 completion(s) pending 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower760: running 1 completion(s) 09:50:05,251 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 4.679 ms 09:50:05,251 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower760: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower760: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@4479d7a6 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Number of entries to be appended = 1 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Append entry to log one 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower760: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Log size is now 2 09:50:05,251 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 325.8 μs 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower760, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached -1 (of expected -1) 09:50:05,251 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower760: awaiting more completions to resolve 1 deferred callback(s) 09:50:05,251 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: 1, nextIndex: 2 09:50:05,251 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:05,252 AM [leader759-writer-162] [DEBUG] EntryStoreCompleter - leader759: 1 completion(s) pending 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 2.141 ms 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - leader759: running 1 completion(s) 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 5.980 μs 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader759: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader759 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader759 (Leader): Setting last applied to 1 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 101.7 μs 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from 0 to 1 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting to receive commands 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader759: Applying state for log index 1 data one 09:50:05,252 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader759: applyState called: one 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 188.3 μs 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 81.56 μs 09:50:05,252 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting to receive commands 09:50:05,253 AM [follower760-writer-160] [DEBUG] EntryStoreCompleter - follower760: 1 completion(s) pending 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower760: running 1 completion(s) 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 1.573 ms 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower760: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower760: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@2699bb6e 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Commit index set to 0 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 0 (of expected -1) 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): Setting last applied to 0 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from -1 to 0 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower760: Applying state for log index 0 data zero 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower760: applyState called: zero 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 339.5 μs 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): Commit index set to 1 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower760 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 82.80 μs 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): Setting last applied to 1 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from 0 to 1 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower760: Applying state for log index 1 data one 09:50:05,253 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower760: applyState called: one 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 114.0 μs 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 62.77 μs 09:50:05,253 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands [WARN] [06/30/2025 09:50:05.253] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower760] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower760, initialSyncDone=true] [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.254] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower760] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,254 AM [test-pekko.actor.default-dispatcher-5] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,343 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:50:05,343 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:50:05,462 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:50:05,463 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:50:05,463 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:50:05,496 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries: new entries applied - resuming follower follower761 09:50:05,583 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:50:05,583 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:50:05,583 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:50:05,584 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): Number of entries to be appended = 2 09:50:05,584 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:05,584 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): Append entry to log zero 09:50:05,584 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:05,584 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:50:05,584 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 1 commands after 844.8 ms 09:50:05,584 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): Append entry to log one 09:50:05,584 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): Log size is now 2 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): Commit index set to 1 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower761, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:05,585 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: 1, nextIndex: 2 09:50:05,585 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - leader759: fakeSnapshot purging log to 0 for term 6 09:50:05,585 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower761 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 0 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to 0 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower761: Applying state for log index 0 data zero 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - follower761: applyState called: zero 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 1 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from 0 to 1 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower761: Applying state for log index 1 data one 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - follower761: applyState called: one 09:50:05,585 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower761: awaiting more completions to resolve 2 deferred callback(s) 09:50:05,589 AM [follower761-writer-161] [DEBUG] EntryStoreCompleter - follower761: 1 completion(s) pending 09:50:05,589 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower761: running 1 completion(s) 09:50:05,589 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 4.821 ms 09:50:05,589 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 2 commands after 3.720 μs 09:50:05,589 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:05,589 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower761: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@7f18885d 09:50:05,589 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower761: awaiting more completions to resolve 1 deferred callback(s) 09:50:05,591 AM [follower761-writer-161] [DEBUG] EntryStoreCompleter - follower761: 1 completion(s) pending 09:50:05,591 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: completed 2 commands in 2.362 ms 09:50:05,591 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: waiting to receive commands 09:50:05,591 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower761: running 1 completion(s) 09:50:05,591 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:05,591 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower761: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@e8bbb2e [WARN] [06/30/2025 09:50:05.591] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=true] [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.592] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,592 AM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,703 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:50:05,703 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower761: fakeSnapshot purging log to 0 for term 6 09:50:05,703 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:50:05,703 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower760: fakeSnapshot purging log to 0 for term 6 09:50:05,748 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries complete 09:50:05,748 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower760-collector762#1409032591] 09:50:05,759 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower760] 09:50:05,760 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PekkoRaftStorage - follower760: terminating thread VirtualThread[#901,follower760-writer-160]/waiting 09:50:05,760 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 506.3 ms 09:50:05,760 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 91.95 μs 09:50:05,760 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: journal writer stopped 09:50:05,760 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower760: journal closed 09:50:05,760 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower760: journal closed 09:50:05,760 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower760: stopped executor with 0 remaining tasks 09:50:05,760 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower760: stopped executor with 0 remaining tasks 09:50:05,760 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower761-collector763#1470418043] 09:50:05,761 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower761] 09:50:05,761 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PekkoRaftStorage - follower761: terminating thread VirtualThread[#905,follower761-writer-161]/waiting 09:50:05,761 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 1 commands after 169.9 ms 09:50:05,761 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 11.52 μs 09:50:05,761 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: journal writer stopped 09:50:05,761 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower761: journal closed 09:50:05,761 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower761: journal closed 09:50:05,761 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks 09:50:05,761 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks 09:50:05,761 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader759-collector764#-508383542] 09:50:05,762 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader759] 09:50:05,762 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PekkoRaftStorage - leader759: terminating thread VirtualThread[#906,leader759-writer-162]/waiting 09:50:05,762 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 509.6 ms 09:50:05,762 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 11.11 μs 09:50:05,762 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: journal writer stopped 09:50:05,762 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader759: journal closed 09:50:05,762 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - leader759: journal closed 09:50:05,762 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: stopped executor with 0 remaining tasks 09:50:05,762 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader759: stopped executor with 0 remaining tasks 09:50:05,764 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaInstallSnapshot starting [WARN] [06/30/2025 09:50:05.766] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower769]: org.opendaylight.controller.cluster.common.actor.Monitor@2e2a52c5 [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.766] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower769] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,766 AM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower769] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,767 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - follower769: Starting recovery with journal batch size 1000 09:50:05,767 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower769: started executor 09:50:05,767 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower769: started executor 09:50:05,767 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower769: metadata journal open: replayFrom=1 applyTo=0 09:50:05,768 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower769: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:05,768 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower769: journal open: applyTo=0 09:50:05,768 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: journal writer started 09:50:05,768 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands [WARN] [06/30/2025 09:50:05.769] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower770]: org.opendaylight.controller.cluster.common.actor.Monitor@3f9891b2 [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.769] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,770 AM [test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,770 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/follower769/TermInfo.properties does not exist 09:50:05,770 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/follower769/journal-v1-1.log 09:50:05,770 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/follower769/metadata-v1-1.log 09:50:05,770 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - follower770: Starting recovery with journal batch size 1000 09:50:05,770 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower769: no eligible files found 09:50:05,770 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower770: started executor 09:50:05,770 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower769: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:05,770 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower770: started executor 09:50:05,770 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower770: metadata journal open: replayFrom=1 applyTo=0 09:50:05,771 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower770: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:05,771 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower770: journal open: applyTo=0 09:50:05,771 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: journal writer started 09:50:05,771 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:50:05,772 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/follower770/TermInfo.properties does not exist 09:50:05,772 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/follower770/journal-v1-1.log 09:50:05,772 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/follower770/metadata-v1-1.log 09:50:05,772 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower770: no eligible files found 09:50:05,772 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower770: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:05,774 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=0, votedFor=null 09:50:05,774 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower769: Local TermInfo store seeded with TermInfo{term=0} 09:50:05,774 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower769: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@656143 09:50:05,774 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from -1 to -1 09:50:05,775 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - follower769: journal recovery completedin 387.6 μs with journalIndex=1 09:50:05,776 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=0, votedFor=null 09:50:05,776 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower770: Local TermInfo store seeded with TermInfo{term=0} 09:50:05,776 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower770: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2f9dfcbc 09:50:05,776 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to -1 09:50:05,777 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - follower770: journal recovery completedin 358.5 μs with journalIndex=1 [WARN] [06/30/2025 09:50:05.871] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@700befa5 [INFO] [pekkoDeadLetter][06/30/2025 09:50:05.871] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader768] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,871 AM [test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader768] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:50:05,871 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 09:50:05,871 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader768: started executor 09:50:05,872 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader768: started executor 09:50:05,872 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=0 09:50:05,873 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:05,873 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=0 09:50:05,873 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: journal writer started 09:50:05,873 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:05,874 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties does not exist 09:50:05,874 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:05,874 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:05,874 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:05,874 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:50:05,874 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:05,877 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:50:05,877 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader768: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader768} 09:50:05,877 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@435b504c 09:50:05,877 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 09:50:05,877 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - leader768: journal recovery completedin 163.6 μs with journalIndex=1 09:50:06,383 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:50:06,383 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:06,383 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 09:50:06,383 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 09:50:06,387 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 09:50:06,387 AM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader768 (Candidate): Starting new election term 6 09:50:06,387 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} to peer follower769 09:50:06,388 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} to peer follower770 09:50:06,388 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:06,388 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:50:06,388 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 09:50:06,388 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:06,388 AM [test-pekko.actor.default-dispatcher-7] [INFO] Follower - follower769 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:06,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:06,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 09:50:06,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:06,388 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower770 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:06,392 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=6, votedFor=null 09:50:06,392 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:06,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=null 09:50:06,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:06,395 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=6, votedFor=leader768 09:50:06,395 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Election: Leader has following peers: [follower769, follower770] 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:50:06,395 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Last sync leader does not match current leader leader768, need to catch up to -1 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/30/2025 09:50:06.395] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower769] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower769, initialSyncDone=false] 09:50:06,395 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:06,396 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=leader768 09:50:06,396 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:06,396 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower770: Last sync leader does not match current leader leader768, need to catch up to -1 09:50:06,396 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: -1, nextIndex: 0 09:50:06,396 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:06.396] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=false] 09:50:06,432 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=0 term=6 command=zero 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:06,432 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 559.1 ms 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower769 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower770 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:06,432 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:06,432 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:50:06,432 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=1 term=6 command=one 09:50:06,432 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Append entry to log zero 09:50:06,432 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:06,432 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:50:06,432 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:06,433 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Log size is now 1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Append entry to log zero 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower769 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:06,433 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower770 09:50:06,433 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Log size is now 1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:06,433 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 664.4 ms 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 0, nextIndex: 1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 1 is present for follower follower769 09:50:06,433 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 661.9 ms 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:06,433 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower770: awaiting more completions to resolve 1 deferred callback(s) 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 0, nextIndex: 1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 1 is present for follower follower770 09:50:06,433 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:06,437 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:50:06,437 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 5.165 ms 09:50:06,437 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 4.310 μs 09:50:06,437 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:50:06,437 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:06,437 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:50:06,437 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 0 09:50:06,437 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to 0 09:50:06,438 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Applying state for log index 0 data zero 09:50:06,438 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader768: applyState called: zero 09:50:06,438 AM [follower770-writer-164] [DEBUG] EntryStoreCompleter - follower770: 1 completion(s) pending 09:50:06,438 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:06,438 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 5.477 ms 09:50:06,438 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 5.399 ms 09:50:06,438 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:06,438 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower770: running 1 completion(s) 09:50:06,438 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:50:06,438 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:06,438 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:06,438 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:06,438 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower770: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@6fb2838b 09:50:06,438 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@32c58eaf 09:50:06,438 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:06,438 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:06,438 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Append entry to log one 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Append entry to log one 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Log size is now 2 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Log size is now 2 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:06,439 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 342.6 μs 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:50:06,439 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower770: awaiting more completions to resolve 1 deferred callback(s) 09:50:06,439 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:50:06,439 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 368.5 μs 09:50:06,439 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 1, nextIndex: 2 09:50:06,439 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:06,439 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 1, nextIndex: 2 09:50:06,439 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:06,440 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 2.258 ms 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 3.780 μs 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 103.8 μs 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 1 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 0 to 1 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 1 data one 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: one 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 411.4 μs 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 72.86 μs 09:50:06,440 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:06,440 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:06,440 AM [follower770-writer-164] [DEBUG] EntryStoreCompleter - follower770: 1 completion(s) pending 09:50:06,440 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader768: fakeSnapshot purging log to 0 for term 6 09:50:06,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:06,440 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 1.576 ms 09:50:06,440 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower770: running 1 completion(s) 09:50:06,440 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:06,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:06,440 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 1.671 ms 09:50:06,440 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:06,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@34063548 09:50:06,440 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:50:06,440 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower770: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@2bb4b766 09:50:06,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Commit index set to 0 09:50:06,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 0 (of expected -1) 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Commit index set to 0 09:50:06,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 0 (of expected -1) 09:50:06,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 0 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from -1 to 0 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 0 09:50:06,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 0 data zero 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to 0 09:50:06,441 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower769: applyState called: zero 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower770: Applying state for log index 0 data zero 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower770: applyState called: zero 09:50:06,441 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 429.2 μs 09:50:06,441 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 407.8 μs 09:50:06,441 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 80.44 μs 09:50:06,441 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:06,441 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 92.05 μs 09:50:06,441 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands [WARN] [06/30/2025 09:50:06.441] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=true] [WARN] [06/30/2025 09:50:06.441] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower769] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower769, initialSyncDone=true] 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower769 (Follower): Commit index set to 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Commit index set to 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 0 to 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower769: Applying state for log index 1 data one 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 0 to 1 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower769: applyState called: one 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower770: Applying state for log index 1 data one 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - follower770: applyState called: one 09:50:06,441 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower769: fakeSnapshot purging log to 0 for term 6 09:50:06,441 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 441.1 μs 09:50:06,441 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - follower770: fakeSnapshot purging log to 0 for term 6 09:50:06,441 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 54.08 μs 09:50:06,442 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:50:06,441 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 573.2 μs 09:50:06,442 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 85.02 μs 09:50:06,442 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:06,633 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:06,634 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:06,753 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:06,873 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:06,993 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader768: Persist data index=2 term=6 command=two 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:07,89 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 648.5 ms 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader768: Persist data index=3 term=6 command=three 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader768: Persist data index=4 term=6 command=four 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Append entry to log two 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader768: Persist data index=5 term=6 command=five 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Log size is now 2 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:07,89 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 647.6 ms 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 2, nextIndex: 3 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 3, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:50:07,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 3 is present for follower follower769 09:50:07,90 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}]} 09:50:07,90 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Persist data index=6 term=6 command=six 09:50:07,90 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:50:07,90 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true 09:50:07,90 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:50:07,90 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 3 is present for follower follower769 09:50:07,90 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:50:07,92 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:07,92 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:50:07,92 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:07,92 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 2.974 ms 09:50:07,92 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:07,92 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 3.449 ms 09:50:07,92 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 09:50:07,92 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 4 commands after 4.140 μs 09:50:07,92 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@67199d2b 09:50:07,92 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:50:07,92 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:07,92 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}]} 09:50:07,92 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 3 09:50:07,92 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:50:07,92 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Append entry to log three 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 2 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 1 to 2 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader768: Applying state for log index 2 data two 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader768: applyState called: two 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Append entry to log four 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:50:07,93 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 266.0 μs 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Append entry to log five 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Log size is now 5 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=5, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:50:07,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 3 deferred callback(s) 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader768: fakeSnapshot purging log to 1 for term 6 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 5, nextIndex: 6 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 1 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 6 is present for follower follower769 09:50:07,93 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=5, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:50:07,94 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:07,95 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:07,95 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 1.914 ms 09:50:07,95 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 09:50:07,95 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@350da305 09:50:07,95 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 2 commands after 4.770 μs 09:50:07,95 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 2 deferred callback(s) 09:50:07,97 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 2 completion(s) pending 09:50:07,97 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 2 commands in 2.872 ms 09:50:07,98 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: running 2 completion(s) 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false} 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@6a898ab5 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false} 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@69518331 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=5, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Append entry to log six 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Log size is now 6 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Commit index set to 2 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 2 (of expected -1) 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 6 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 2 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 1 to 2 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower769: Applying state for log index 2 data two 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower769: applyState called: two 09:50:07,98 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 4 completion(s) pending 09:50:07,98 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 745.9 μs 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower769: fakeSnapshot purging log to 1 for term 6 09:50:07,98 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 4 commands in 5.944 ms 09:50:07,98 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 2.770 μs 09:50:07,98 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:50:07,98 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader768: running 4 completion(s) 09:50:07,98 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 81.27 μs 09:50:07,98 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:07,98 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:07,99 AM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader768: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=4, mandatoryTrim=false] 09:50:07,99 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#909,leader768-writer-165]/waiting 09:50:07,100 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 1.077 ms 09:50:07,100 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 8.700 μs 09:50:07,100 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: journal writer stopped 09:50:07,100 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader768: journal closed 09:50:07,100 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader768: journal closed 09:50:07,100 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:50:07,100 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:50:07,100 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:07,100 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 2.124 ms [ERROR] [06/30/2025 09:50:07.099] [test-pekko.actor.internal-dispatcher-2] [pekko://test/user/leader768] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:287) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:07,100 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 4.030 μs [WARN] [06/30/2025 09:50:07.101] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@fa585dc 09:50:07,101 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:07,101 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false} 09:50:07,101 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 100.2 μs 09:50:07,101 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:07,101 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@1bc96ac9 09:50:07,101 AM [test-pekko.actor.default-dispatcher-15] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:287) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:07,101 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 09:50:07,101 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: started executor 09:50:07,101 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: started executor 09:50:07,102 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=3 09:50:07,104 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=7 replayFrom=1 fileEntries=0 09:50:07,104 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=3 09:50:07,104 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: journal writer started 09:50:07,104 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRecovery - leader768: restoring local TermInfo{term=6, votedFor=leader768} 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@6c5a4984 09:50:07,105 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 09:50:07,106 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:50:07,106 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:50:07,106 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:50:07,107 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:50:07,107 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:50:07,107 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:50:07,107 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:50:07,107 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader768: journal recovery completedin 1.890 ms with journalIndex=1 09:50:07,189 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,189 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,189 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,189 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,290 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,290 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,290 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,290 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,390 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,390 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,390 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,390 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,491 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,491 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,491 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,491 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,533 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:07,533 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 09:50:07,533 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader768 (Candidate): Starting new election term 7 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} to peer follower769 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} to peer follower770 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 09:50:07,539 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:07,539 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 09:50:07,539 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:07,539 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower769 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 09:50:07,539 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower770 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:50:07,543 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=7, votedFor=null 09:50:07,543 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 6 09:50:07,544 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=7, votedFor=null 09:50:07,544 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 6, lastTerm: 6 09:50:07,546 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=7, votedFor=leader768 09:50:07,546 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [INFO] Candidate - leader768 (Candidate): LastApplied index -1 is behind last index 6 - switching to PreLeader 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Election: Leader has following peers: [follower769, follower770] 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:50:07,547 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Persist data index=7 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:07,548 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=7, votedFor=leader768 09:50:07,548 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Replicate message: identifier: null, logIndex: 7, isSendImmediate: true 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:07,548 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: received 1 commands after 443.9 ms 09:50:07,548 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:07,548 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 6, nextIndex: 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 7 is present for follower follower769 09:50:07,548 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Sending AppendEntries to follower follower769: AppendEntries{term=7, leaderId=leader768, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader768, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 7 is present for follower follower769 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Log size is now 6 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower769, logLastIndex=7, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:50:07,549 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 7, nextIndex: 8 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:07,549 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 448.3 ms 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:50:07,549 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:07,551 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:07,551 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 1.660 ms 09:50:07,551 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:07,551 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:07,551 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=false} 09:50:07,551 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@1b3ac2a8 09:50:07,551 AM [leader768-writer-166] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:50:07,551 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 2.825 ms 09:50:07,551 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:07,551 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:50:07,551 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:07,551 AM [test-pekko.actor.default-dispatcher-12] [INFO] SnapshotManager - leader768: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=7, lastTerm=7, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=8, mandatoryTrim=false] 09:50:07,551 AM [test-pekko.actor.default-dispatcher-6] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:07,551 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#910,leader768-writer-166]/waiting [ERROR] [06/30/2025 09:50:07.551] [test-pekko.actor.internal-dispatcher-2] [pekko://test/user/leader768] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:07,552 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: received 1 commands after 917.2 μs 09:50:07,552 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 20.25 μs 09:50:07,552 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: journal writer stopped 09:50:07,552 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader768: journal closed 09:50:07,552 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader768: journal closed 09:50:07,552 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:50:07,552 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:50:07.552] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@62bfd514 09:50:07,553 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 09:50:07,553 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: started executor 09:50:07,553 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: started executor 09:50:07,554 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=3 09:50:07,555 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=8 replayFrom=1 fileEntries=0 09:50:07,555 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=3 09:50:07,555 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: journal writer started 09:50:07,555 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:07,556 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 09:50:07,556 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,556 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,556 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRecovery - leader768: restoring local TermInfo{term=7, votedFor=leader768} 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@590d0f89 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:50:07,557 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:50:07,558 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:50:07,558 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:50:07,558 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:50:07,558 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:50:07,559 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387] 09:50:07,559 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: journal recovery completedin 2.024 ms with journalIndex=1 09:50:07,591 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,591 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,591 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,592 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,692 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,692 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,692 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,692 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,793 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,793 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,793 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,793 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,893 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,893 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,893 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,893 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:07,994 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:07,994 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:07,994 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:07,994 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,43 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:08,43 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 09:50:08,44 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 09:50:08,48 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=8, votedFor=leader768 09:50:08,48 AM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader768 (Candidate): Starting new election term 8 09:50:08,48 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} to peer follower769 09:50:08,48 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} to peer follower770 09:50:08,48 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 8 09:50:08,48 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:08,48 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:08,48 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 09:50:08,48 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 09:50:08,48 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 09:50:08,49 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 09:50:08,49 AM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower769 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 09:50:08,49 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower770 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 09:50:08,52 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=8, votedFor=null 09:50:08,52 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 6 09:50:08,53 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=8, votedFor=null 09:50:08,53 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 7, lastTerm: 7 09:50:08,55 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=8, votedFor=leader768 09:50:08,55 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=8, voteGranted=true}, my term is 8 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=8, voteGranted=true}, current voteCount: 1 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [INFO] Candidate - leader768 (Candidate): LastApplied index -1 is behind last index 7 - switching to PreLeader 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Election: Leader has following peers: [follower769, follower770] 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:08,55 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Persist data index=8 term=8 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:08,56 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:08,56 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Replicate message: identifier: null, logIndex: 8, isSendImmediate: true 09:50:08,56 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:08,56 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:08,56 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 commands after 500.8 ms 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=8, votedFor=leader768 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 7, nextIndex: 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 8 is present for follower follower769 09:50:08,56 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader768 (PreLeader): Sending AppendEntries to follower follower769: AppendEntries{term=8, leaderId=leader768, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=8, leaderId=leader768, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:08,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:08,57 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Log size is now 7 09:50:08,57 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=8, success=true, followerId=follower769, logLastIndex=8, logLastTerm=8, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:08,57 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:50:08,57 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 505.9 ms 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 8 is present for follower follower769 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 8, nextIndex: 9 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:08,57 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:08,57 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:50:08,58 AM [leader768-writer-167] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:50:08,58 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 1.867 ms 09:50:08,58 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (PreLeader): Applying to log - commitIndex: 8, lastAppliedIndex: -1 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 0 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to 0 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 0 data zero 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: zero 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 1 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 0 to 1 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 1 data one 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: one 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 2 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 1 to 2 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 2 data two 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: two 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 3 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 2 to 3 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 3 data three 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: three 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 4 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 3 to 4 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 4 data four 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: four 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 5 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 4 to 5 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 5 data five 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: five 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 6 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 5 to 6 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 6 data six 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader768: applyState called: six 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 7 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 6 to 7 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 7 to 8 09:50:08,58 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader768: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:08,59 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:50:08,59 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PreLeader - leader768 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 8, lastIndex: 8 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=false} 09:50:08,59 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 2.004 ms 09:50:08,59 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): Election: Leader has following peers: [follower769, follower770] 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@5f6c9930 09:50:08,59 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:08,59 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:08,59 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 commands after 954.4 μs 09:50:08,59 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:08,59 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader768 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 8 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Commit index set to 8 09:50:08,59 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 101.5 μs 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,59 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 2, lastIndex(): 8 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 3 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 2 to 3 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 3 data three 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower769: applyState called: three 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 4 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 3 to 4 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 4 data four 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower769: applyState called: four 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 5 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 4 to 5 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 5 data five 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower769: applyState called: five 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 6 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 5 to 6 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 6 data six 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower769: applyState called: six 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 7 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 6 to 7 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 8 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 7 to 8 09:50:08,59 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower769: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:08,59 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 615.3 μs 09:50:08,59 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 101.0 μs 09:50:08,59 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:50:08,95 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,95 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,95 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,95 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,173 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,196 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,196 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,196 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,196 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,294 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,296 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,296 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,296 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,296 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,397 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,397 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,397 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,397 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,413 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,497 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,497 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,497 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,497 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,533 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,598 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,598 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,598 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,598 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,653 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,773 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,800 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,800 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,800 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:08,893 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:08,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:08,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:08,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:08,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,1 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,1 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,1 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,1 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,13 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,133 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,202 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,202 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,202 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,202 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,253 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,373 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,493 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,613 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,733 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,805 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,805 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,805 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,805 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,853 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:09,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:09,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:09,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:09,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:09,973 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,6 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,6 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,6 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,6 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,207 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,207 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,207 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,207 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,213 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,333 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,408 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,408 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,408 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,409 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,453 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,573 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,693 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,813 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:10,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:10,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:10,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:10,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:10,933 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,12 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,12 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,12 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,12 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,53 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,113 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,113 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,113 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,113 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,173 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,293 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,314 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,314 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,314 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,314 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,413 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,515 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,515 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,515 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,515 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,533 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,615 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,616 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,616 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,616 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,653 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,716 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,716 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,716 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,716 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,774 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,817 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,817 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,817 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,817 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:11,893 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:11,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:11,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:11,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:11,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:12,13 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:50:12,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/TermInfo.properties 09:50:12,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/journal-v1-1.log 09:50:12,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12066592677790318490/leader768/metadata-v1-1.log 09:50:12,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:50:12,119 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower769-collector771#-1371116746] 09:50:12,130 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower769] 09:50:12,130 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - follower769: terminating thread VirtualThread[#907,follower769-writer-163]/waiting 09:50:12,130 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 4.071 s 09:50:12,130 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 12.83 μs 09:50:12,130 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: journal writer stopped 09:50:12,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower769: journal closed 09:50:12,131 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - follower769: journal closed 09:50:12,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower769: stopped executor with 0 remaining tasks 09:50:12,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower769: stopped executor with 0 remaining tasks 09:50:12,131 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower770-collector772#-1906938547] 09:50:12,131 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower770] 09:50:12,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - follower770: terminating thread VirtualThread[#908,follower770-writer-164]/waiting 09:50:12,131 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 5.690 s 09:50:12,132 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 7.690 μs 09:50:12,132 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: journal writer stopped 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower770: journal closed 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - follower770: journal closed 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks 09:50:12,132 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader768-collector773#381238768] 09:50:12,132 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader768] 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#912,leader768-writer-167]/waiting 09:50:12,132 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 commands after 4.073 s 09:50:12,132 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 8.920 μs 09:50:12,132 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: journal writer stopped 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader768: journal closed 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader768: journal closed 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:50:12,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:50:12,135 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaAppendEntries starting [WARN] [06/30/2025 09:50:12.136] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower778]: org.opendaylight.controller.cluster.common.actor.Monitor@1fbbee8a 09:50:12,137 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - follower778: Starting recovery with journal batch size 1000 09:50:12,137 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower778: started executor 09:50:12,138 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower778: started executor 09:50:12,138 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower778: metadata journal open: replayFrom=1 applyTo=0 09:50:12,139 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower778: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:12,139 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - follower778: journal open: applyTo=0 09:50:12,139 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: journal writer started 09:50:12,139 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:12,140 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/follower778/TermInfo.properties does not exist 09:50:12,140 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/follower778/journal-v1-1.log 09:50:12,140 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/follower778/metadata-v1-1.log 09:50:12,140 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower778: no eligible files found 09:50:12,140 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower778: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:50:12.140] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower779]: org.opendaylight.controller.cluster.common.actor.Monitor@2a523fa3 09:50:12,141 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - follower779: Starting recovery with journal batch size 1000 09:50:12,141 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower779: started executor 09:50:12,142 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower779: started executor 09:50:12,143 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower779: metadata journal open: replayFrom=1 applyTo=0 09:50:12,143 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=0, votedFor=null 09:50:12,143 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower778: Local TermInfo store seeded with TermInfo{term=0} 09:50:12,143 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower778: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@19608034 09:50:12,143 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from -1 to -1 09:50:12,143 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower779: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:12,143 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower779: journal open: applyTo=0 09:50:12,144 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: journal writer started 09:50:12,144 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:50:12,144 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - follower778: journal recovery completedin 280.4 μs with journalIndex=1 [WARN] [06/30/2025 09:50:12.145] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader777]: org.opendaylight.controller.cluster.common.actor.Monitor@3f2d536a 09:50:12,146 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader777: Starting recovery with journal batch size 1000 09:50:12,146 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: started executor 09:50:12,146 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: started executor 09:50:12,146 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader777: metadata journal open: replayFrom=1 applyTo=0 09:50:12,146 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/follower779/TermInfo.properties does not exist 09:50:12,146 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/follower779/journal-v1-1.log 09:50:12,146 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/follower779/metadata-v1-1.log 09:50:12,146 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: no eligible files found 09:50:12,146 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower779: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:12,147 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader777: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:12,147 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader777: journal open: applyTo=0 09:50:12,147 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: journal writer started 09:50:12,147 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:12,148 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties does not exist 09:50:12,148 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:12,148 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:12,148 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:12,148 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 09:50:12,148 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader777: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:12,151 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=0, votedFor=null 09:50:12,151 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower779: Local TermInfo store seeded with TermInfo{term=0} 09:50:12,151 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower779: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7a187394 09:50:12,151 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to -1 09:50:12,152 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - follower779: journal recovery completedin 259.4 μs with journalIndex=1 09:50:12,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 09:50:12,152 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader777: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader777} 09:50:12,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@49a8c789 09:50:12,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to -1 09:50:12,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: journal recovery completedin 212.8 μs with journalIndex=1 09:50:12,583 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:50:12,583 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader777 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:12,583 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader777 (Follower): Received ElectionTimeout - switching to Candidate 09:50:12,583 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): Election: Candidate has following voting peers: [follower779, follower778] 09:50:12,589 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 09:50:12,589 AM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader777 (Candidate): Starting new election term 6 09:50:12,590 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} to peer follower779 09:50:12,590 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} to peer follower778 09:50:12,590 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader777 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:50:12,590 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:12,590 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:12,590 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): Cluster state: [] 09:50:12,590 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: [] 09:50:12,590 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:12,590 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:12,590 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower779 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:12,590 AM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower778 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:12,595 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=null 09:50:12,595 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:12,596 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=6, votedFor=null 09:50:12,596 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): In requestVote: RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=leader777 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Election: Leader has following peers: [follower779, follower778] 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader777 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:50:12,598 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Last sync leader does not match current leader leader777, need to catch up to -1 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: -1, nextIndex: 0 09:50:12,598 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:12.599] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=false] 09:50:12,599 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=6, votedFor=leader777 09:50:12,599 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:12,599 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Last sync leader does not match current leader leader777, need to catch up to -1 09:50:12,599 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: -1, nextIndex: 0 09:50:12,599 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:12.599] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower778] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower778, initialSyncDone=false] 09:50:12,606 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader777: Persist data index=0 term=6 command=zero 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower779 09:50:12,607 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 459.9 ms 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Append entry to log zero 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Log size is now 1 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:50:12,607 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 463.6 ms 09:50:12,607 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: awaiting more completions to resolve 1 deferred callback(s) 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:12,607 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower778 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 0, nextIndex: 1 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader777: Persist data index=1 term=6 command=one 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 1 is present for follower follower779 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:12,608 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower778 09:50:12,608 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:12,608 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:50:12,608 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:12,609 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Append entry to log zero 09:50:12,609 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:12,609 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Log size is now 1 09:50:12,609 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:12,609 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:50:12,609 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:50:12,609 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 469.8 ms 09:50:12,609 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 0, nextIndex: 1 09:50:12,609 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:12,609 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 1 is present for follower follower778 09:50:12,609 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:12,612 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:50:12,612 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 5.471 ms 09:50:12,612 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 3.840 μs 09:50:12,612 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:50:12,612 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:12,612 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:50:12,612 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 0 09:50:12,612 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to 0 09:50:12,613 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 0 data zero 09:50:12,613 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: zero 09:50:12,614 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:50:12,614 AM [follower779-writer-169] [DEBUG] EntryStoreCompleter - follower779: 1 completion(s) pending 09:50:12,614 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 6.293 ms 09:50:12,614 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:50:12,614 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 4.872 ms 09:50:12,614 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: running 1 completion(s) 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@7ec32124 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@70263246 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Append entry to log one 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Append entry to log one 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Log size is now 2 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:12,614 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 418.3 μs 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:50:12,614 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: awaiting more completions to resolve 1 deferred callback(s) 09:50:12,614 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 1, nextIndex: 2 09:50:12,614 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 557.7 μs 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Log size is now 2 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:50:12,614 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:50:12,615 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.191 ms 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 6.560 μs 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 90.35 μs 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 1 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 0 to 1 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader777: Applying state for log index 1 data one 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader777: applyState called: one 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 443.6 μs 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 75.97 μs 09:50:12,615 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader777: fakeSnapshot purging log to 0 for term 6 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 1, nextIndex: 2 09:50:12,615 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:50:12,616 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:50:12,616 AM [follower779-writer-169] [DEBUG] EntryStoreCompleter - follower779: 1 completion(s) pending 09:50:12,616 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 1.763 ms 09:50:12,616 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:12,616 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 1.962 ms 09:50:12,616 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: running 1 completion(s) 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@277a66df 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Commit index set to 0 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@6ff48188 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 0 (of expected -1) 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 0 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to 0 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower779: Applying state for log index 0 data zero 09:50:12,616 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower779: applyState called: zero 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Commit index set to 1 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 0 09:50:12,616 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from -1 to 0 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower778: Applying state for log index 0 data zero 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower778: applyState called: zero 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 1 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 0 to 1 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower778: Applying state for log index 1 data one 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower778: applyState called: one 09:50:12,617 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 677.9 μs 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Commit index set to 1 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 1 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 0 to 1 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower779: Applying state for log index 1 data one 09:50:12,617 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 76.67 μs 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower779: applyState called: one 09:50:12,617 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:12,617 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 634.4 μs 09:50:12,617 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower779: fakeSnapshot purging log to 0 for term 6 09:50:12,617 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 37.43 μs 09:50:12,617 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 1.670 μs 09:50:12,617 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 23.40 μs 09:50:12,617 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:50:12,617 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower778: fakeSnapshot purging log to 0 for term 6 [WARN] [06/30/2025 09:50:12.617] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=true] [WARN] [06/30/2025 09:50:12.617] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower778] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower778, initialSyncDone=true] 09:50:12,833 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:50:12,833 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Persist data index=2 term=6 command=two 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:12,858 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 243.0 ms 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower778 09:50:12,858 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Append entry to log two 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Log size is now 2 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:50:12,859 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 241.9 ms 09:50:12,859 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:50:12,859 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 2, nextIndex: 3 09:50:12,859 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:12,861 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:50:12,861 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.577 ms 09:50:12,861 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:12,861 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:12,861 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 2.270 ms 09:50:12,861 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:50:12,861 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:50:12,861 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 2 09:50:12,861 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@5bb4be53 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 1 to 2 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 2 data two 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader777: applyState called: two 09:50:12,861 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader777: fakeSnapshot purging log to 1 for term 6 09:50:12,861 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 449.7 μs 09:50:12,862 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 115.1 μs 09:50:12,862 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:12,953 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:50:12,953 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}]} 09:50:13,73 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:50:13,73 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}]} 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Commit index set to 2 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 2 (of expected -1) 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 2 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 1 to 2 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower778: Applying state for log index 2 data two 09:50:13,73 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower778: applyState called: two 09:50:13,74 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower778: fakeSnapshot purging log to 1 for term 6 09:50:13,74 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 212.6 ms 09:50:13,74 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 104.7 μs 09:50:13,74 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Persist data index=3 term=6 command=three 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 09:50:13,110 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 248.2 ms 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 3 is present for follower follower778 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]} 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]} 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Append entry to log three 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Log size is now 2 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 2 (of expected -1) 09:50:13,110 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 36.42 ms 09:50:13,110 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:50:13,111 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 3, nextIndex: 4 09:50:13,111 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:50:13,112 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:50:13,112 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:50:13,112 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.489 ms 09:50:13,112 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:13,112 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 2.056 ms 09:50:13,112 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:13,112 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:50:13,112 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:50:13,112 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:13,112 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 09:50:13,113 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@48e51eba 09:50:13,113 AM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader777: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false] 09:50:13,113 AM [test-pekko.actor.default-dispatcher-15] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [ERROR] [06/30/2025 09:50:13.113] [test-pekko.actor.internal-dispatcher-3] [pekko://test/user/leader777] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:13,114 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader777: terminating thread VirtualThread[#915,leader777-writer-170]/waiting 09:50:13,114 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 1.460 ms 09:50:13,114 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 14.85 μs 09:50:13,114 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: journal writer stopped 09:50:13,114 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader777: journal closed 09:50:13,114 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader777: journal closed 09:50:13,114 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:50:13,114 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:50:13.115] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader777]: org.opendaylight.controller.cluster.common.actor.Monitor@6565eb38 09:50:13,116 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader777: Starting recovery with journal batch size 1000 09:50:13,116 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: started executor 09:50:13,116 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: started executor 09:50:13,117 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader777: metadata journal open: replayFrom=1 applyTo=3 09:50:13,118 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader777: journal open: firstIndex=1 lastIndex=4 replayFrom=1 fileEntries=0 09:50:13,118 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader777: journal open: applyTo=3 09:50:13,118 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: journal writer started 09:50:13,118 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader777: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader777: restoring local TermInfo{term=6, votedFor=leader777} 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@62dfa63e 09:50:13,121 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to -1 09:50:13,122 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:50:13,122 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:50:13,123 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:50:13,123 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:50:13,123 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: journal recovery completedin 1.683 ms with journalIndex=1 09:50:13,211 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,211 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,211 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,211 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,311 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,311 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,311 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,311 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,412 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,412 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,412 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,412 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,512 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,512 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,512 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,512 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,613 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,613 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,613 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,613 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,633 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader777 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:13,633 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader777 (Follower): Received ElectionTimeout - switching to Candidate 09:50:13,633 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): Election: Candidate has following voting peers: [follower779, follower778] 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=7, votedFor=leader777 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader777 (Candidate): Starting new election term 7 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} to peer follower779 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} to peer follower778 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader777 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 09:50:13,638 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:13,638 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: [] 09:50:13,638 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} with higher term appears reachable, updating term. 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:13,638 AM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower779 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Cluster state: [] 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} with higher term appears reachable, updating term. 09:50:13,638 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower778 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:50:13,641 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=7, votedFor=null 09:50:13,642 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): In requestVote: RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 3, lastTerm: 6 09:50:13,642 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=7, votedFor=null 09:50:13,642 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 6 09:50:13,646 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=7, votedFor=leader777 09:50:13,646 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:50:13,646 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 09:50:13,646 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 09:50:13,646 AM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader777 (Candidate): LastApplied index -1 is behind last index 3 - switching to PreLeader 09:50:13,646 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Election: Leader has following peers: [follower779, follower778] 09:50:13,646 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:50:13,647 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader777 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 09:50:13,647 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=7, votedFor=leader777 09:50:13,647 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 3, nextIndex: 4 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Persist data index=4 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:13,647 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:13,648 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 commands after 529.8 ms 09:50:13,648 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Replicate message: identifier: null, logIndex: 4, isSendImmediate: true 09:50:13,648 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:50:13,648 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:50:13,648 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): sendAppendEntries: 4 is present for follower follower778 09:50:13,648 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Sending AppendEntries to follower follower778: AppendEntries{term=7, leaderId=leader777, prevLogIndex=3, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader777, prevLogIndex=3, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): Log size is now 3 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower778, logLastIndex=4, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 09:50:13,648 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:50:13,648 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 536.0 ms 09:50:13,649 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 4, nextIndex: 5 09:50:13,649 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:13,649 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:13,649 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:13,649 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:13,649 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:50:13,649 AM [leader777-writer-171] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:50:13,649 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 1.753 ms 09:50:13,649 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:13,649 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Applying to log - commitIndex: 4, lastAppliedIndex: -1 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 0 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to 0 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 0 data zero 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader777: applyState called: zero 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 1 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 0 to 1 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 1 data one 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader777: applyState called: one 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 2 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 1 to 2 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 2 data two 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader777: applyState called: two 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 3 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 2 to 3 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 3 data three 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader777: applyState called: three 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 4 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 3 to 4 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 4 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:13,650 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 commands after 523.9 μs 09:50:13,650 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 64.07 μs 09:50:13,650 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PreLeader - leader777 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 4, lastIndex: 4 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Election: Leader has following peers: [follower779, follower778] 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:50:13,650 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader777 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 7 09:50:13,650 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=false} 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@23f7bbaf 09:50:13,650 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 1.888 ms 09:50:13,650 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): Commit index set to 4 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 3 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 2 to 3 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower778: Applying state for log index 3 data three 09:50:13,650 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower778: applyState called: three 09:50:13,651 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 4 09:50:13,651 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 3 to 4 09:50:13,651 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower778: Applying state for log index 4 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:13,651 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 146.7 μs 09:50:13,651 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 105.0 μs 09:50:13,651 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:50:13,713 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,713 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,713 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,713 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,763 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:13,814 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,814 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,814 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,814 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:13,883 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:13,914 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:13,914 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:13,914 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:13,914 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,3 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,15 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,15 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,15 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,15 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,115 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,115 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,115 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,115 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,123 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,216 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,216 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,216 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,216 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,243 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,316 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,316 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,316 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,316 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,363 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,417 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,417 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,417 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,417 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,483 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,517 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,517 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,517 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,517 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,603 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,618 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,618 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,618 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,618 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,718 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,718 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,718 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,718 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,723 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,819 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,819 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,819 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,819 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,843 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:14,919 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:14,919 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:14,919 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:14,919 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:14,963 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,20 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,20 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,20 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,20 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,83 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,120 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,120 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,120 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,120 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,203 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,221 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,221 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,221 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,221 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,321 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,321 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,321 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,321 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,323 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,422 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,422 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,422 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,422 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,443 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,522 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,522 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,522 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,522 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,563 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,623 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,623 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,623 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,623 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,684 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,723 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,723 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,723 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,723 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,803 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,824 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,824 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,824 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,824 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:15,924 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:15,925 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:15,925 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:15,925 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:15,925 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,25 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,25 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,25 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,25 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,43 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,126 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,126 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,126 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,126 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,163 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,226 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,226 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,226 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,226 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,283 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,327 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,327 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,327 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,327 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,404 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,428 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,428 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,428 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,428 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,524 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,528 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,528 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,528 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,528 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,629 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,629 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,629 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,629 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,643 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,729 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,729 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,730 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,730 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,764 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,830 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,830 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,830 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,830 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:16,883 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:16,931 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:16,931 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:16,931 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:16,931 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,3 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,31 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,31 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,31 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,31 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,123 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,132 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,132 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,132 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,132 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,232 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,233 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,233 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,233 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,243 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,333 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,333 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,333 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,333 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,363 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,433 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,434 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,434 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,434 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,483 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,534 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,534 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,534 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,534 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,603 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,635 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,635 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,635 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,635 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,723 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,735 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,735 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,735 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,735 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,836 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,836 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,836 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,836 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,842 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:17,936 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:17,936 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:17,936 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:17,936 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:17,963 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:18,37 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/TermInfo.properties 09:50:18,37 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/journal-v1-1.log 09:50:18,37 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14216108078829559278/leader777/metadata-v1-1.log 09:50:18,37 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:50:18,83 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:50:18,138 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower778-collector780#791536466] 09:50:18,149 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower778] 09:50:18,150 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower778: terminating thread VirtualThread[#913,follower778-writer-168]/waiting 09:50:18,150 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 4.499 s 09:50:18,150 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 16.30 μs 09:50:18,150 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: journal writer stopped 09:50:18,150 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower778: journal closed 09:50:18,150 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower778: journal closed 09:50:18,150 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower778: stopped executor with 0 remaining tasks 09:50:18,150 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower778: stopped executor with 0 remaining tasks 09:50:18,150 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower779-collector781#830016043] 09:50:18,151 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower779] 09:50:18,151 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower779: terminating thread VirtualThread[#914,follower779-writer-169]/waiting 09:50:18,151 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 5.534 s 09:50:18,151 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 11.62 μs 09:50:18,151 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: journal writer stopped 09:50:18,151 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower779: journal closed 09:50:18,151 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower779: journal closed 09:50:18,151 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks 09:50:18,151 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks 09:50:18,151 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader777-collector782#677946084] 09:50:18,152 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader777] 09:50:18,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader777: terminating thread VirtualThread[#916,leader777-writer-171]/waiting 09:50:18,152 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 commands after 4.502 s 09:50:18,152 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 5.591 μs 09:50:18,152 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: journal writer stopped 09:50:18,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader777: journal closed 09:50:18,152 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader777: journal closed 09:50:18,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:50:18,152 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:50:18,154 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderInstallsSnapshotWithRestartedFollowerDuringSnapshotInstallation starting [WARN] [06/30/2025 09:50:18.155] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower787]: org.opendaylight.controller.cluster.common.actor.Monitor@127fd7f5 09:50:18,156 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower787: Starting recovery with journal batch size 1000 09:50:18,157 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower787: started executor 09:50:18,157 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower787: started executor 09:50:18,157 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower787: metadata journal open: replayFrom=1 applyTo=0 09:50:18,158 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower787: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:18,158 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower787: journal open: applyTo=0 09:50:18,158 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: journal writer started 09:50:18,158 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:18,159 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/follower787/TermInfo.properties does not exist 09:50:18,159 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/follower787/journal-v1-1.log 09:50:18,159 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/follower787/metadata-v1-1.log 09:50:18,159 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: no eligible files found 09:50:18,160 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower787: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:50:18.161] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower788]: org.opendaylight.controller.cluster.common.actor.Monitor@118f492f 09:50:18,162 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower788: Starting recovery with journal batch size 1000 09:50:18,162 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower788: started executor 09:50:18,162 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower788: started executor 09:50:18,163 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower788: metadata journal open: replayFrom=1 applyTo=0 09:50:18,164 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower788: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:18,164 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower788: journal open: applyTo=0 09:50:18,164 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: journal writer started 09:50:18,164 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:50:18,164 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/follower788/TermInfo.properties does not exist 09:50:18,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=0, votedFor=null 09:50:18,165 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower787: Local TermInfo store seeded with TermInfo{term=0} 09:50:18,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/follower788/journal-v1-1.log 09:50:18,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower787: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3c1ff9cb 09:50:18,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/follower788/metadata-v1-1.log 09:50:18,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from -1 to -1 09:50:18,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: no eligible files found 09:50:18,165 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower788: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:18,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower787: journal recovery completedin 182.6 μs with journalIndex=1 [WARN] [06/30/2025 09:50:18.165] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@7a54e92 09:50:18,166 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 09:50:18,166 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: started executor 09:50:18,166 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: started executor 09:50:18,166 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=0 09:50:18,167 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:18,167 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=0 09:50:18,167 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: journal writer started 09:50:18,167 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:18,168 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties does not exist 09:50:18,168 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:18,168 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:18,168 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:18,168 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:50:18,168 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:18,169 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=0, votedFor=null 09:50:18,169 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower788: Local TermInfo store seeded with TermInfo{term=0} 09:50:18,169 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower788: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@cb934dc 09:50:18,169 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to -1 09:50:18,169 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - follower788: journal recovery completedin 183.7 μs with journalIndex=1 09:50:18,170 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:50:18,170 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader786: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader786} 09:50:18,170 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@4d7e48f4 09:50:18,170 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 09:50:18,170 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader786: journal recovery completedin 158.7 μs with journalIndex=1 09:50:18,623 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:50:18,623 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:18,623 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 09:50:18,623 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 09:50:18,628 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 09:50:18,628 AM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader786 (Candidate): Starting new election term 6 09:50:18,629 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} to peer follower788 09:50:18,629 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} to peer follower787 09:50:18,629 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:50:18,629 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:18,629 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Cluster state: [] 09:50:18,629 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:18,629 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower788 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:18,629 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:18,629 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 09:50:18,629 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:18,629 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower787 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:18,633 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=null 09:50:18,633 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): In requestVote: RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:18,633 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=6, votedFor=null 09:50:18,633 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:18,636 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=leader786 09:50:18,636 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:18,636 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:50:18,636 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:50:18,636 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Election: Leader has following peers: [follower788, follower787] 09:50:18,636 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:18,636 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:18,636 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:50:18,637 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Last sync leader does not match current leader leader786, need to catch up to -1 09:50:18,637 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: -1, nextIndex: 0 09:50:18,637 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:18.637] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=false] 09:50:18,637 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=6, votedFor=leader786 09:50:18,637 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:18,637 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Last sync leader does not match current leader leader786, need to catch up to -1 09:50:18,637 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: -1, nextIndex: 0 09:50:18,637 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:18.637] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower787] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower787, initialSyncDone=false] 09:50:18,679 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=0 term=6 command=zero 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower788 09:50:18,680 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 512.9 ms 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower787 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=1 term=6 command=one 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower788 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower787 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:18,680 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Append entry to log zero 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:18,680 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Append entry to log zero 09:50:18,680 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Log size is now 1 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:50:18,680 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: awaiting more completions to resolve 1 deferred callback(s) 09:50:18,680 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 516.8 ms 09:50:18,681 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 522.5 ms 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 0, nextIndex: 1 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 1 is present for follower follower788 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:18,681 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Log size is now 1 09:50:18,681 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:18,681 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:50:18,681 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 0, nextIndex: 1 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 1 is present for follower follower787 09:50:18,681 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:18,685 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:50:18,685 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 5.020 ms 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:50:18,685 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 5.190 μs 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 0 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to 0 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 0 data zero 09:50:18,685 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: zero 09:50:18,686 AM [follower788-writer-173] [DEBUG] EntryStoreCompleter - follower788: 1 completion(s) pending 09:50:18,686 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:50:18,686 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 5.513 ms 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:50:18,686 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 5.435 ms 09:50:18,686 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:50:18,686 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@663eb534 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: running 1 completion(s) 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@75b13644 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Append entry to log one 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Append entry to log one 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Log size is now 2 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Log size is now 2 09:50:18,686 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:18,686 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:18,686 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 473.9 μs 09:50:18,687 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:50:18,687 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:50:18,687 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:50:18,687 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: awaiting more completions to resolve 1 deferred callback(s) 09:50:18,687 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 613.1 μs 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 1, nextIndex: 2 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 1, nextIndex: 2 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:18,687 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:50:18,687 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 2.034 ms 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:18,687 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 3.190 μs 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 1 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 0 to 1 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Applying state for log index 1 data one 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - leader786: applyState called: one 09:50:18,687 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 154.3 μs 09:50:18,687 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - leader786: fakeSnapshot purging log to 0 for term 6 09:50:18,687 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 2.810 μs 09:50:18,687 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 47.13 μs 09:50:18,687 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:18,688 AM [follower788-writer-173] [DEBUG] EntryStoreCompleter - follower788: 1 completion(s) pending 09:50:18,688 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:50:18,688 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 1.398 ms 09:50:18,688 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:50:18,688 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 1.262 ms 09:50:18,688 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: running 1 completion(s) 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@7c4ff7a0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@4f91af37 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Commit index set to 0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Commit index set to 0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 0 (of expected -1) 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 0 (of expected -1) 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to 0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from -1 to 0 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower788: Applying state for log index 0 data zero 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 0 data zero 09:50:18,688 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower788: applyState called: zero 09:50:18,688 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: zero 09:50:18,688 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 481.2 μs 09:50:18,688 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower788 (Follower): Commit index set to 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:18,688 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 509.1 μs 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 0 to 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower788: Applying state for log index 1 data one 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower788: applyState called: one 09:50:18,689 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 80.78 μs 09:50:18,689 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:18,689 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower788: fakeSnapshot purging log to 0 for term 6 09:50:18,689 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 80.07 μs 09:50:18,689 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 3.230 μs 09:50:18,689 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 43.34 μs 09:50:18,689 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands [WARN] [06/30/2025 09:50:18.689] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=true] 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Commit index set to 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 0 to 1 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 1 data one 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: one 09:50:18,689 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower787: fakeSnapshot purging log to 0 for term 6 09:50:18,689 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 480.1 μs [WARN] [06/30/2025 09:50:18.689] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower787] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower787, initialSyncDone=true] 09:50:18,689 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 102.8 μs 09:50:18,689 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:18,873 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:18,873 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:18,882 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - follower788: terminating thread VirtualThread[#919,follower788-writer-173]/waiting 09:50:18,882 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 193.1 ms 09:50:18,882 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 13.80 μs 09:50:18,882 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: journal writer stopped 09:50:18,882 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower788: journal closed 09:50:18,882 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower788: journal closed 09:50:18,882 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks 09:50:18,882 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:50:18.993] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:18,993 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) [WARN] [06/30/2025 09:50:19.113] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,113 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:19,233 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) [WARN] [06/30/2025 09:50:19.233] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=2 term=6 command=two 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:19,287 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 599.6 ms 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=3 term=6 command=three 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=4 term=6 command=four 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true 09:50:19,287 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 [WARN] [06/30/2025 09:50:19.287] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=5 term=6 command=five [WARN] [06/30/2025 09:50:19.288] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} [WARN] [06/30/2025 09:50:19.288] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log two [WARN] [06/30/2025 09:50:19.288] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Log size is now 2 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=6 term=6 command=six 09:50:19,288 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true 09:50:19,288 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 598.7 ms 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 [WARN] [06/30/2025 09:50:19.288] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:50:19,288 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 2, nextIndex: 3 09:50:19,288 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:50:19,288 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 3 is present for follower follower787 09:50:19,288 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:50:19,290 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:50:19,290 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:50:19,290 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 3.456 ms 09:50:19,291 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 4 commands after 4.750 μs 09:50:19,291 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 2.481 ms 09:50:19,291 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 2 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 1 to 2 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 2 data two 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: two 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@558b8ed8 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 4 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log three 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log four 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log five 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log six 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Log size is now 6 09:50:19,291 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 2 commands after 732.3 μs 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:19,291 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:50:19,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 4 deferred callback(s) 09:50:19,292 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader786: fakeSnapshot purging log to 1 for term 6 09:50:19,292 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 6, nextIndex: 7 09:50:19,292 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1 09:50:19,295 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 2 completion(s) pending 09:50:19,295 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 2 commands in 3.248 ms 09:50:19,295 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: running 2 completion(s) 09:50:19,295 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 2 commands after 3.760 μs 09:50:19,295 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 09:50:19,295 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@425233ef 09:50:19,295 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false} 09:50:19,295 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@7119bfa7 09:50:19,295 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 2 deferred callback(s) 09:50:19,296 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 4 completion(s) pending 09:50:19,296 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 4 commands in 5.738 ms 09:50:19,296 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader786: running 4 completion(s) 09:50:19,296 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:50:19,296 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 4.630 μs 09:50:19,296 AM [test-pekko.actor.default-dispatcher-12] [INFO] SnapshotManager - leader786: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=4, mandatoryTrim=false] 09:50:19,296 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 144.9 μs 09:50:19,297 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:19,297 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 2 completion(s) pending 09:50:19,297 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 2 commands in 2.363 ms 09:50:19,297 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:19,297 AM [test-pekko.actor.default-dispatcher-6] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [ERROR] [06/30/2025 09:50:19.297] [test-pekko.actor.internal-dispatcher-11] [pekko://test/user/leader786] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:19,297 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#920,leader786-writer-174]/waiting 09:50:19,297 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: running 2 completion(s) 09:50:19,297 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false} 09:50:19,297 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@3ed42137 09:50:19,297 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false} 09:50:19,297 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@5d8e446b 09:50:19,298 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 977.4 μs 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Commit index set to 2 09:50:19,298 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 9.930 μs 09:50:19,298 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: journal writer stopped 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 2 (of expected -1) 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 6 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 2 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 1 to 2 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower787: Applying state for log index 2 data two 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower787: applyState called: two 09:50:19,298 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader786: journal closed 09:50:19,298 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader786: journal closed 09:50:19,298 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:50:19,298 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:50:19,298 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower787: fakeSnapshot purging log to 1 for term 6 09:50:19,298 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 750.6 μs 09:50:19,298 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 66.42 μs 09:50:19,298 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands [WARN] [06/30/2025 09:50:19.298] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@4fbc992f 09:50:19,299 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 09:50:19,299 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: started executor 09:50:19,299 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: started executor 09:50:19,299 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=3 09:50:19,300 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=7 replayFrom=1 fileEntries=0 09:50:19,301 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=3 09:50:19,301 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: journal writer started 09:50:19,301 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader786: restoring local TermInfo{term=6, votedFor=leader786} 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2d0ae97b 09:50:19,302 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 09:50:19,303 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:50:19,303 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:50:19,303 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:50:19,303 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:50:19,304 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:50:19,304 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:50:19,304 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:50:19,304 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: journal recovery completedin 1.628 ms with journalIndex=1 09:50:19,388 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,388 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,388 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,388 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,488 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,488 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,488 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,488 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,589 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,589 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,589 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,589 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,689 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,689 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,689 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,689 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,790 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,790 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,790 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,790 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,803 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:19,804 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 09:50:19,804 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 09:50:19,807 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 09:50:19,807 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader786 (Candidate): Starting new election term 7 09:50:19,807 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} to peer follower788 09:50:19,808 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} to peer follower787 09:50:19,808 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 [WARN] [06/30/2025 09:50:19.808] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} 09:50:19,808 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:19,808 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 09:50:19,808 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 09:50:19,808 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower787 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:50:19,811 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=7, votedFor=null 09:50:19,811 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 6, lastTerm: 6 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=7, votedFor=leader786 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader786 (Candidate): LastApplied index -1 is behind last index 6 - switching to PreLeader 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Election: Leader has following peers: [follower788, follower787] 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) [WARN] [06/30/2025 09:50:19.814] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=7, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=7 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Replicate message: identifier: null, logIndex: 7, isSendImmediate: true 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:19,814 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: received 1 commands after 513.6 ms 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 6, nextIndex: 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 7 is present for follower follower787 09:50:19,814 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Sending AppendEntries to follower follower787: AppendEntries{term=7, leaderId=leader786, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} [WARN] [06/30/2025 09:50:19.814] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=7, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader786, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Log size is now 6 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower787, logLastIndex=7, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:50:19,815 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 7 is present for follower follower787 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 7, nextIndex: 8 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:50:19,815 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:19,815 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 517.4 ms 09:50:19,817 AM [leader786-writer-175] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:50:19,817 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 2.617 ms 09:50:19,817 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:50:19,817 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:19,817 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:19,817 AM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader786: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=7, lastTerm=7, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=8, mandatoryTrim=false] 09:50:19,818 AM [test-pekko.actor.default-dispatcher-12] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) [ERROR] [06/30/2025 09:50:19.817] [test-pekko.actor.internal-dispatcher-11] [pekko://test/user/leader786] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:139) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:50:19,818 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:50:19,818 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:50:19,818 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 2.561 ms 09:50:19,818 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:19,818 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=false} 09:50:19,818 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@60ab5b7 09:50:19,818 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#921,leader786-writer-175]/waiting 09:50:19,818 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: received 1 commands after 1.293 ms 09:50:19,818 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 8.450 μs 09:50:19,818 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: journal writer stopped 09:50:19,818 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader786: journal closed 09:50:19,818 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader786: journal closed 09:50:19,818 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:50:19,819 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:50:19.819] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@141a0d04 09:50:19,820 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 09:50:19,820 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: started executor 09:50:19,820 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: started executor 09:50:19,821 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=3 09:50:19,822 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=8 replayFrom=1 fileEntries=0 09:50:19,822 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=3 09:50:19,822 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: journal writer started 09:50:19,822 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:19,823 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 09:50:19,823 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,823 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader786: restoring local TermInfo{term=7, votedFor=leader786} 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@12ecdd2 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:50:19,824 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:50:19,825 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:50:19,825 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:50:19,825 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:50:19,825 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387] 09:50:19,825 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: journal recovery completedin 1.437 ms with journalIndex=1 09:50:19,890 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,890 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,890 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,890 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:19,991 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:19,991 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:19,991 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:19,991 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,92 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,92 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,92 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,92 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,192 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,192 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,192 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,192 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,293 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:20,293 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 09:50:20,293 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,293 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,293 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 09:50:20,293 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,293 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,296 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=8, votedFor=leader786 09:50:20,296 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader786 (Candidate): Starting new election term 8 09:50:20,297 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} to peer follower788 09:50:20,297 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} to peer follower787 09:50:20,297 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 8 09:50:20,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:20,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 09:50:20,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 09:50:20,297 AM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower787 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term [WARN] [06/30/2025 09:50:20.298] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} 09:50:20,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=8, votedFor=null 09:50:20,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 7, lastTerm: 7 09:50:20,301 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=8, votedFor=leader786 09:50:20,301 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=8, voteGranted=true}, my term is 8 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=8, voteGranted=true}, current voteCount: 1 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader786 (Candidate): LastApplied index -1 is behind last index 7 - switching to PreLeader 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Election: Leader has following peers: [follower788, follower787] 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 8 [WARN] [06/30/2025 09:50:20.302] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=8 term=8 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:20,302 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Replicate message: identifier: null, logIndex: 8, isSendImmediate: true 09:50:20,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:20,303 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 commands after 480.6 ms 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 7, nextIndex: 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 8 is present for follower follower787 09:50:20,303 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Sending AppendEntries to follower follower787: AppendEntries{term=8, leaderId=leader786, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} [WARN] [06/30/2025 09:50:20.303] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=8, leaderId=leader786, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true}]} 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower787 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower787 (Follower): Log size is now 7 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=8, success=true, followerId=follower787, logLastIndex=8, logLastTerm=8, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:50:20,303 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:50:20,303 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 485.5 ms 09:50:20,303 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 8 is present for follower follower787 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 8, nextIndex: 9 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:20,304 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:20,305 AM [leader786-writer-176] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:50:20,305 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 2.126 ms 09:50:20,305 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=true} 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Applying to log - commitIndex: 8, lastAppliedIndex: -1 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 0 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to 0 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 0 data zero 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: zero 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 1 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 0 to 1 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 1 data one 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: one 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 2 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 1 to 2 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 2 data two 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: two 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 3 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 2 to 3 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 3 data three 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: three 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 4 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 3 to 4 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 4 data four 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: four 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 5 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 4 to 5 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 5 data five 09:50:20,305 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: five 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 6 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 5 to 6 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 6 data six 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: six 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 7 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 6 to 7 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 8 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 7 to 8 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:20,306 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:50:20,306 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 2.177 ms 09:50:20,306 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:20,306 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PreLeader - leader786 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 8, lastIndex: 8 09:50:20,306 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387, pending=false} 09:50:20,306 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 commands after 976.6 μs 09:50:20,306 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@7060e438 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Election: Leader has following peers: [follower788, follower787] 09:50:20,306 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 100.7 μs 09:50:20,306 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:50:20,306 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader786 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 8 [WARN] [06/30/2025 09:50:20.306] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): Commit index set to 8 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 2, lastIndex(): 8 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 3 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 2 to 3 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower787: Applying state for log index 3 data three 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower787: applyState called: three 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 4 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 3 to 4 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower787: Applying state for log index 4 data four 09:50:20,306 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower787: applyState called: four 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 5 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 4 to 5 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower787: Applying state for log index 5 data five 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower787: applyState called: five 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 6 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 5 to 6 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower787: Applying state for log index 6 data six 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower787: applyState called: six 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 7 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 6 to 7 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower787: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 8 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 7 to 8 09:50:20,307 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower787: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@40666387 09:50:20,307 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 1.147 ms 09:50:20,307 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 83.29 μs 09:50:20,307 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:50:20,393 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,393 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,393 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,393 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,423 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:20.423] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,494 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,494 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,494 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,494 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,543 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:20.543] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,594 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,594 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,594 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,594 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,663 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:20.663] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,695 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,695 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,695 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,695 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:20.783] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,783 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:20,796 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,796 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,796 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,796 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:20,896 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,896 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,896 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,896 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:20.903] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:20,903 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:20,997 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:20,997 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:20,997 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:20,997 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:21,24 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:21.024] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,97 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,97 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,97 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,97 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.143] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,143 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,198 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,198 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,198 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,198 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.263] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,263 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,298 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,299 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,299 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,299 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.383] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,383 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,399 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,399 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,399 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,399 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:21,500 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,500 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,500 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,500 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.503] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,503 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,600 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,600 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,600 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,600 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.623] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,624 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,701 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,701 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,701 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,701 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.743] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,743 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:21.864] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:21,864 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:21,902 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:21,902 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:21,902 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:21,902 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:21,983 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:21.983] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:22,103 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,103 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,103 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,103 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:22,103 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:22.103] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.223] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,223 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:22,304 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,304 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,304 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,304 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.343] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,343 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:22,404 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,404 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,404 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,404 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.463] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,463 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:22,505 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,505 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,505 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,505 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.583] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,583 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:22,606 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,606 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,606 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,606 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.703] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,703 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:22,706 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,706 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,706 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,706 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:22,807 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,807 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,807 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,807 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.813] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,813 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:22,907 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:22,907 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:22,907 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:22,907 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:22.923] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:22,923 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,8 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,8 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,8 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,8 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.033] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,33 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,109 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,109 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,109 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,109 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.153] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,153 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,209 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,209 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,209 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,209 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.263] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,263 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,310 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,310 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,310 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,310 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.383] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,383 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,410 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,410 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,410 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,410 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.493] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,493 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,511 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,511 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,511 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,511 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:23,612 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,612 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,612 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,612 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.613] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,614 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,712 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,712 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,712 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,712 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.733] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,733 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,813 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,813 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,813 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,813 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.853] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,853 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:23,913 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:23,913 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:23,913 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:23,913 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:23.973] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:23,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:24,14 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:24,14 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:24,14 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:24,14 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:50:24.093] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:24,93 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:50:24,114 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:24,114 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:24,114 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:24,114 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:24,213 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:50:24.213] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:50:24,215 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/TermInfo.properties 09:50:24,215 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/journal-v1-1.log 09:50:24,215 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest6358303372182354258/leader786/metadata-v1-1.log 09:50:24,215 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:50:24,316 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower787-collector789#452623030] 09:50:24,327 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower787] 09:50:24,327 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - follower787: terminating thread VirtualThread[#918,follower787-writer-172]/waiting 09:50:24,327 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 4.020 s 09:50:24,327 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 25.86 μs 09:50:24,327 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: journal writer stopped 09:50:24,328 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower787: journal closed 09:50:24,328 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower787: journal closed 09:50:24,328 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower787: stopped executor with 0 remaining tasks 09:50:24,328 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower787: stopped executor with 0 remaining tasks 09:50:24,328 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower788-collector790#1944440273] 09:50:24,328 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower788] 09:50:24,328 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader786-collector791#363463285] 09:50:24,329 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader786] 09:50:24,329 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#923,leader786-writer-176]/waiting 09:50:24,329 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 commands after 4.023 s 09:50:24,329 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 5.950 μs 09:50:24,329 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: journal writer stopped 09:50:24,329 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader786: journal closed 09:50:24,329 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader786: journal closed 09:50:24,329 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:50:24,329 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:50:24,332 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotTriggeredByMemoryThresholdExceededWithLaggingFollower starting [WARN] [06/30/2025 09:50:24.333] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower796]: org.opendaylight.controller.cluster.common.actor.Monitor@50ca949f 09:50:24,334 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower796: Starting recovery with journal batch size 1000 09:50:24,334 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: started executor 09:50:24,334 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: started executor 09:50:24,335 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower796: metadata journal open: replayFrom=1 applyTo=0 09:50:24,335 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower796: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:24,335 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower796: journal open: applyTo=0 09:50:24,336 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: journal writer started 09:50:24,336 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:50:24,337 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/follower796/TermInfo.properties does not exist 09:50:24,337 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/follower796/journal-v1-1.log 09:50:24,337 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/follower796/metadata-v1-1.log 09:50:24,337 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: no eligible files found 09:50:24,337 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower796: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:50:24.337] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower797]: org.opendaylight.controller.cluster.common.actor.Monitor@1bced21 09:50:24,338 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - follower797: Starting recovery with journal batch size 1000 09:50:24,338 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower797: started executor 09:50:24,338 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower797: started executor 09:50:24,338 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower797: metadata journal open: replayFrom=1 applyTo=0 09:50:24,339 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower797: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:24,339 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower797: journal open: applyTo=0 09:50:24,339 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: journal writer started 09:50:24,339 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting to receive commands 09:50:24,340 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/follower797/TermInfo.properties does not exist 09:50:24,340 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/follower797/journal-v1-1.log 09:50:24,340 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/follower797/metadata-v1-1.log 09:50:24,340 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: no eligible files found 09:50:24,340 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower797: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:50:24.340] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader795]: org.opendaylight.controller.cluster.common.actor.Monitor@5e4190ae 09:50:24,341 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader795: Starting recovery with journal batch size 1000 09:50:24,341 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader795: started executor 09:50:24,341 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader795: started executor 09:50:24,341 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader795: metadata journal open: replayFrom=1 applyTo=0 09:50:24,341 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader795: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:50:24,341 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader795: journal open: applyTo=0 09:50:24,342 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: journal writer started 09:50:24,342 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:50:24,343 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/leader795/TermInfo.properties does not exist 09:50:24,343 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader795: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/leader795/journal-v1-1.log 09:50:24,343 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader795: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15711337493253306907/leader795/metadata-v1-1.log 09:50:24,343 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader795: no eligible files found 09:50:24,343 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=5, votedFor=leader795 09:50:24,343 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader795: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:50:24,345 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=0, votedFor=null 09:50:24,345 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower796: Local TermInfo store seeded with TermInfo{term=0} 09:50:24,345 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower796: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3ed82999 09:50:24,345 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from -1 to -1 09:50:24,346 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower796: journal recovery completedin 147.9 μs with journalIndex=1 09:50:24,349 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=0, votedFor=null 09:50:24,349 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower797: Local TermInfo store seeded with TermInfo{term=0} 09:50:24,349 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower797: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@6ead77eb 09:50:24,349 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower797: Moving last applied index from -1 to -1 09:50:24,349 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - follower797: journal recovery completedin 173.2 μs with journalIndex=1 09:50:24,350 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=5, votedFor=leader795 09:50:24,350 AM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader795: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader795} 09:50:24,350 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader795: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@362c32c3 09:50:24,350 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from -1 to -1 09:50:24,350 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - leader795: journal recovery completedin 335.9 μs with journalIndex=1 09:50:24,813 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:50:24,813 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - leader795 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.12:17355, Removed) 09:50:24,813 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - leader795 (Follower): Received ElectionTimeout - switching to Candidate 09:50:24,813 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader795 (Candidate): Election: Candidate has following voting peers: [follower797, follower796] 09:50:24,816 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=6, votedFor=leader795 09:50:24,816 AM [test-pekko.actor.default-dispatcher-7] [INFO] Candidate - leader795 (Candidate): Starting new election term 6 09:50:24,816 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader795 (Candidate): Sending RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} to peer follower797 09:50:24,816 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader795 (Candidate): Sending RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} to peer follower796 09:50:24,816 AM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader795 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:50:24,816 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:24,816 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): Cluster state: [] 09:50:24,816 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:24,816 AM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower797 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:24,816 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower796 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:50:24,817 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower796 (Follower): Cluster state: [] 09:50:24,817 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower796 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:50:24,817 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower796 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:50:24,819 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=6, votedFor=null 09:50:24,819 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): In requestVote: RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:24,820 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=6, votedFor=null 09:50:24,820 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower796 (Follower): In requestVote: RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:50:24,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=6, votedFor=leader795 09:50:24,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:50:24,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader795 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:50:24,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader795 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:50:24,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Election: Leader has following peers: [follower797, follower796] 09:50:24,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:24,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:24,822 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader795 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:50:24,822 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower797: Last sync leader does not match current leader leader795, need to catch up to -1 09:50:24,822 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower797 updated: matchIndex: -1, nextIndex: 0 09:50:24,822 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:50:24,822 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=6, votedFor=leader795 09:50:24,822 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower796 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} [WARN] [06/30/2025 09:50:24.822] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower797] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower797, initialSyncDone=false] 09:50:24,822 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Last sync leader does not match current leader leader795, need to catch up to -1 09:50:24,822 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: -1, nextIndex: 0 09:50:24,822 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:50:24.822] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower796] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower796, initialSyncDone=false] 09:50:24,843 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:50:24,843 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader795: Persist data index=0 term=6 command=zero 09:50:24,843 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader795: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:24,843 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:50:24,843 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:24,843 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 0 is present for follower follower797 09:50:24,843 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 501.9 ms 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower797: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 0 is present for follower follower796 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower796: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Number of entries to be appended = 1 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Append entry to log zero 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower796: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:24,844 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 508.3 ms 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Log size is now 1 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower796, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:24,844 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: 0, nextIndex: 1 09:50:24,844 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached -1 (of expected -1) 09:50:24,844 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:50:24,844 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: awaiting more completions to resolve 1 deferred callback(s) 09:50:24,844 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): Number of entries to be appended = 1 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): Append entry to log zero 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower797: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): Log size is now 1 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower797, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached -1 (of expected -1) 09:50:24,845 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 commands after 505.8 ms 09:50:24,845 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower797: awaiting more completions to resolve 1 deferred callback(s) 09:50:24,845 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower797 updated: matchIndex: 0, nextIndex: 1 09:50:24,845 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:50:24,848 AM [leader795-writer-179] [DEBUG] EntryStoreCompleter - leader795: 1 completion(s) pending 09:50:24,848 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 4.758 ms 09:50:24,848 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader795: running 1 completion(s) 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader795: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - leader795 (Leader): Setting last applied to 0 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from -1 to 0 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader795: Applying state for log index 0 data zero 09:50:24,848 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - leader795: applyState called: zero 09:50:24,849 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 314.7 μs 09:50:24,849 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 91.99 μs 09:50:24,849 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:50:24,849 AM [follower797-writer-178] [DEBUG] EntryStoreCompleter - follower797: 1 completion(s) pending 09:50:24,849 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: completed 1 commands in 4.472 ms 09:50:24,849 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower797: running 1 completion(s) 09:50:24,849 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting to receive commands 09:50:24,849 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower797: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:24,849 AM [follower796-writer-177] [DEBUG] EntryStoreCompleter - follower796: 1 completion(s) pending 09:50:24,849 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower797: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@3357cc6e 09:50:24,849 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 5.257 ms 09:50:24,849 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:50:24,849 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: running 1 completion(s) 09:50:24,849 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower796: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:50:24,849 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@48d2b56b [WARN] [06/30/2025 09:50:24.849] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower797] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower797, initialSyncDone=true] [WARN] [06/30/2025 09:50:24.849] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower796] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower796, initialSyncDone=true] 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower796 (Follower): Commit index set to 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): Commit index set to 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower796 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): Setting last applied to 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower797: Moving last applied index from -1 to 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower796 (Follower): Setting last applied to 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from -1 to 0 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower797: Applying state for log index 0 data zero 09:50:25,54 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower797: applyState called: zero 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower796: Applying state for log index 0 data zero 09:50:25,54 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower796: applyState called: zero 09:50:25,54 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 commands after 204.8 ms 09:50:25,54 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: completed 1 commands in 145.0 μs 09:50:25,54 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting to receive commands 09:50:25,54 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 205.0 ms 09:50:25,54 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 146.5 μs 09:50:25,54 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:50:25,173 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,173 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,293 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,413 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,533 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,602 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader795: Persist data index=1 term=6 command=one 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader795: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:25,603 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 753.9 ms 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 1 is present for follower follower796 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower796: AppendEntries{term=6, leaderId=leader795, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Number of entries to be appended = 1 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Append entry to log one 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower796: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Log size is now 2 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower796, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:50:25,603 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: awaiting more completions to resolve 1 deferred callback(s) 09:50:25,603 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 548.6 ms 09:50:25,603 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: 1, nextIndex: 2 09:50:25,603 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:50:25,605 AM [leader795-writer-179] [DEBUG] EntryStoreCompleter - leader795: 1 completion(s) pending 09:50:25,605 AM [follower796-writer-177] [DEBUG] EntryStoreCompleter - follower796: 1 completion(s) pending 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: running 1 completion(s) 09:50:25,605 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 2.064 ms 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower796: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:50:25,605 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 1.660 ms 09:50:25,605 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:50:25,605 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0882d58@15cba44f 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader795: running 1 completion(s) 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader795: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader795 (Leader): Setting last applied to 1 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from 0 to 1 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader795: Applying state for log index 1 data one 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader795: applyState called: one 09:50:25,605 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader795: fakeSnapshot purging log to 0 for term 6 09:50:25,605 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 406.5 μs 09:50:25,605 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 122.3 μs 09:50:25,605 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower796 (Follower): Commit index set to 1 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower796 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower796 (Follower): Setting last applied to 1 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from 0 to 1 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower796: Applying state for log index 1 data one 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower796: applyState called: one 09:50:25,773 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower796: fakeSnapshot purging log to 0 for term 6 09:50:25,773 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 168.1 ms 09:50:25,773 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 136.2 μs 09:50:25,773 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:50:25,893 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,13 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,134 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,254 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,374 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,493 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,613 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,733 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,853 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:26,973 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,93 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,213 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,333 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,443 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,563 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,683 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,803 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:27,923 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,33 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,153 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,273 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,393 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,513 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,633 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,743 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,863 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:28,983 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,93 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,213 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,323 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,563 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,673 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,783 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:29,893 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,3 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,123 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,232 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,343 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,453 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,563 AM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:50:30,653 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower796-collector798#-1335100086] 09:50:30,664 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower796] 09:50:30,664 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - follower796: terminating thread VirtualThread[#924,follower796-writer-177]/waiting 09:50:30,664 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 4.891 s 09:50:30,665 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 13.55 μs 09:50:30,665 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: journal writer stopped 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower796: journal closed 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower796: journal closed 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: stopped executor with 0 remaining tasks 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower796: stopped executor with 0 remaining tasks 09:50:30,665 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower797-collector799#1693708956] 09:50:30,665 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower797] 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - follower797: terminating thread VirtualThread[#925,follower797-writer-178]/waiting 09:50:30,665 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 commands after 5.611 s 09:50:30,665 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: completed 1 commands in 8.940 μs 09:50:30,665 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: journal writer stopped 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower797: journal closed 09:50:30,665 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower797: journal closed 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower797: stopped executor with 0 remaining tasks 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower797: stopped executor with 0 remaining tasks 09:50:30,666 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader795-collector800#-250133254] 09:50:30,666 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader795] 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader795: terminating thread VirtualThread[#926,leader795-writer-179]/waiting 09:50:30,666 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 5.061 s 09:50:30,666 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 8.560 μs 09:50:30,666 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: journal writer stopped 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader795: journal closed 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader795: journal closed 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader795: stopped executor with 0 remaining tasks 09:50:30,666 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader795: stopped executor with 0 remaining tasks [INFO] [06/30/2025 09:50:30.669] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 09:50:30,670 AM [test-pekko.actor.default-dispatcher-6] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [WARN] [06/30/2025 09:50:30.670] [test-pekko.actor.default-dispatcher-7] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.170.12:17355) [INFO] [06/30/2025 09:50:30.671] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - Exiting completed [INFO] [06/30/2025 09:50:30.671] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - Shutting down... [INFO] [06/30/2025 09:50:30.671] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.12:17355] - Successfully shut down 09:50:30,672 AM [test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - Exiting completed 09:50:30,672 AM [test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - Shutting down... 09:50:30,672 AM [test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.12:17355] - Successfully shut down [INFO] [06/30/2025 09:50:30.672] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.170.12:17355/system/remoting-terminator] Shutting down remote daemon. [INFO] [06/30/2025 09:50:30.673] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.170.12:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports. 09:50:30,673 AM [test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:50:30,673 AM [test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [INFO] [06/30/2025 09:50:30.675] [test-pekko.remote.default-remote-dispatcher-9] [pekko://test@10.30.170.12:17355/system/remoting-terminator] Remoting shut down. 09:50:30,675 AM [test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [DEBUG] [06/30/2025 09:50:30.675] [test-pekko.actor.internal-dispatcher-4] [EventStream] shutting down: StandardOutLogger