[DEBUG] [06/30/2025 09:09:28.942] [main] [EventStream(pekko://test)] logger log1-TestEventListener started 09:09:28,942 AM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started [DEBUG] [06/30/2025 09:09:28.942] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started [DEBUG] [06/30/2025 09:09:28.942] [main] [EventStream(pekko://test)] Default Loggers started [INFO] [06/30/2025 09:09:28.954] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.168:17355] with UID [519541259269125815] 09:09:28,954 AM [test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.168:17355] with UID [519541259269125815] [INFO] [06/30/2025 09:09:28.954] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - Starting up, Pekko version [1.0.3] ... 09:09:28,955 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - Starting up, Pekko version [1.0.3] ... [INFO] [06/30/2025 09:09:28.957] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - Registered cluster JMX MBean [pekko:type=Cluster] [INFO] [06/30/2025 09:09:28.957] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - Started up successfully 09:09:28,957 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 09:09:28,957 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - Started up successfully [INFO] [06/30/2025 09:09:28.962] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing [INFO] [06/30/2025 09:09:28.962] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:28,962 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:28,962 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:28,967 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries starting: sending 2 new payloads [WARN] [06/30/2025 09:09:28.969] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower760]: org.opendaylight.controller.cluster.common.actor.Monitor@1cf72bf7 [INFO] [pekkoDeadLetter][06/30/2025 09:09:28.969] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower760] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:28,970 AM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower760] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:28,971 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - follower760: Starting recovery with journal batch size 1000 09:09:28,971 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower760: started executor 09:09:28,971 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower760: started executor 09:09:28,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower760: metadata journal open: replayFrom=1 applyTo=0 09:09:28,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower760: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:28,972 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower760: journal open: applyTo=0 09:09:28,972 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: journal writer started 09:09:28,972 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands [WARN] [06/30/2025 09:09:28.974] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower761]: org.opendaylight.controller.cluster.common.actor.Monitor@e282c9b 09:09:28,974 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/follower760/TermInfo.properties does not exist 09:09:28,974 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/follower760/journal-v1-1.log 09:09:28,974 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/follower760/metadata-v1-1.log 09:09:28,974 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: no eligible files found 09:09:28,974 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower760: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [INFO] [pekkoDeadLetter][06/30/2025 09:09:28.974] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:28,974 AM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:28,974 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower761: Starting recovery with journal batch size 1000 09:09:28,975 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: started executor 09:09:28,975 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: started executor 09:09:28,975 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower761: metadata journal open: replayFrom=1 applyTo=0 09:09:28,977 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower761: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:28,977 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower761: journal open: applyTo=0 09:09:28,977 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: journal writer started 09:09:28,977 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: waiting to receive commands 09:09:28,978 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/follower761/TermInfo.properties does not exist 09:09:28,978 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/follower761/journal-v1-1.log 09:09:28,978 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/follower761/metadata-v1-1.log 09:09:28,978 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: no eligible files found 09:09:28,978 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower761: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:28,979 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=0, votedFor=null 09:09:28,979 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower760: Local TermInfo store seeded with TermInfo{term=0} 09:09:28,979 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower760: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@5d9689fb 09:09:28,979 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from -1 to -1 09:09:28,979 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - follower760: journal recovery completedin 478.4 μs with journalIndex=1 09:09:28,980 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=0, votedFor=null 09:09:28,980 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower761: Local TermInfo store seeded with TermInfo{term=0} 09:09:28,980 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower761: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3887c796 09:09:28,980 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to -1 09:09:28,980 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower761: journal recovery completedin 179.7 μs with journalIndex=1 09:09:29,75 AM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader759] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [WARN] [06/30/2025 09:09:29.075] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader759]: org.opendaylight.controller.cluster.common.actor.Monitor@751bde68 [INFO] [pekkoDeadLetter][06/30/2025 09:09:29.075] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader759] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,75 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - leader759: Starting recovery with journal batch size 1000 09:09:29,75 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: started executor 09:09:29,75 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: started executor 09:09:29,75 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader759: metadata journal open: replayFrom=1 applyTo=0 09:09:29,76 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader759: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:29,76 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader759: journal open: applyTo=0 09:09:29,76 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: journal writer started 09:09:29,76 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting to receive commands 09:09:29,77 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/leader759/TermInfo.properties does not exist 09:09:29,77 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/leader759/journal-v1-1.log 09:09:29,77 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13357767130558462108/leader759/metadata-v1-1.log 09:09:29,77 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: no eligible files found 09:09:29,77 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=5, votedFor=leader759 09:09:29,77 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader759: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:29,80 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=5, votedFor=leader759 09:09:29,80 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader759: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader759} 09:09:29,80 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader759: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@250d95e 09:09:29,80 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from -1 to -1 09:09:29,80 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader759: journal recovery completedin 133.6 μs with journalIndex=1 09:09:29,588 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:09:29,589 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader759 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:29,589 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader759 (Follower): Received ElectionTimeout - switching to Candidate 09:09:29,589 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader759 (Candidate): Election: Candidate has following voting peers: [follower761, follower760] 09:09:29,592 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=6, votedFor=leader759 09:09:29,592 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader759 (Candidate): Starting new election term 6 09:09:29,592 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader759 (Candidate): Sending RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} to peer follower761 09:09:29,592 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader759 (Candidate): Sending RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} to peer follower760 09:09:29,592 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader759 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:09:29,592 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:29,592 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): Cluster state: [] 09:09:29,592 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:29,592 AM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower760 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:29,593 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower761 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:29,593 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower761 (Follower): Cluster state: [] 09:09:29,593 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower761 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:29,593 AM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower761 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:29,596 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=6, votedFor=null 09:09:29,596 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): In requestVote: RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:29,597 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=null 09:09:29,597 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower761 (Follower): In requestVote: RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:29,599 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=6, votedFor=leader759 09:09:29,599 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:29,599 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader759 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:09:29,599 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader759 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:09:29,599 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader759 (Leader): Election: Leader has following peers: [follower761, follower760] 09:09:29,600 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:29,600 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:29,600 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader759 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:09:29,600 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower760: Last sync leader does not match current leader leader759, need to catch up to -1 09:09:29,600 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/30/2025 09:09:29.600] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower760] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower760, initialSyncDone=false] 09:09:29,600 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:29,600 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=leader759 09:09:29,600 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower761 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} [INFO] [pekkoDeadLetter][06/30/2025 09:09:29.600] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower760] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,600 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower761: Last sync leader does not match current leader leader759, need to catch up to -1 09:09:29,600 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: -1, nextIndex: 0 09:09:29,600 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:29.600] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=false] 09:09:29,601 AM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,601 AM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][06/30/2025 09:09:29.601] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,629 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:09:29,629 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader759: Persist data index=0 term=6 command=zero 09:09:29,629 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader759: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:29,629 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 553.0 ms 09:09:29,629 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower760 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower760: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Number of entries to be appended = 1 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Append entry to log zero 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower760: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Log size is now 1 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower760, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached -1 (of expected -1) 09:09:29,630 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower760: awaiting more completions to resolve 1 deferred callback(s) 09:09:29,630 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 657.8 ms 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: 0, nextIndex: 1 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader759: Persist data index=1 term=6 command=one 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader759: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:29,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:09:29,631 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:29,631 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:09:29,631 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:29,631 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 1 is present for follower follower760 09:09:29,631 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower760: AppendEntries{term=6, leaderId=leader759, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:29,635 AM [leader759-writer-162] [DEBUG] EntryStoreCompleter - leader759: 1 completion(s) pending 09:09:29,635 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 6.015 ms 09:09:29,635 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 3.971 μs 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader759: running 1 completion(s) 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader759: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader759 (Leader): Setting last applied to 0 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from -1 to 0 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader759: Applying state for log index 0 data zero 09:09:29,635 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader759: applyState called: zero 09:09:29,636 AM [follower760-writer-160] [DEBUG] EntryStoreCompleter - follower760: 1 completion(s) pending 09:09:29,636 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 5.722 ms 09:09:29,636 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower760: running 1 completion(s) 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower760: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower760: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@1e1e833c 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Number of entries to be appended = 1 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Append entry to log one 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower760: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Log size is now 2 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower760, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached -1 (of expected -1) 09:09:29,636 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 335.9 μs 09:09:29,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower760: awaiting more completions to resolve 1 deferred callback(s) 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: 1, nextIndex: 2 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:29,637 AM [leader759-writer-162] [DEBUG] EntryStoreCompleter - leader759: 1 completion(s) pending 09:09:29,637 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 2.077 ms 09:09:29,637 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 3.520 μs 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader759: running 1 completion(s) 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader759: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader759 (Leader): Setting last applied to 1 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from 0 to 1 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader759: Applying state for log index 1 data one 09:09:29,637 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader759: applyState called: one 09:09:29,638 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 272.2 μs 09:09:29,638 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting to receive commands 09:09:29,638 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 66.86 μs 09:09:29,638 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 67.30 μs 09:09:29,638 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting to receive commands 09:09:29,638 AM [follower760-writer-160] [DEBUG] EntryStoreCompleter - follower760: 1 completion(s) pending 09:09:29,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower760: running 1 completion(s) 09:09:29,638 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 1.998 ms 09:09:29,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower760: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:29,638 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:09:29,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower760: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@5f238907 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Commit index set to 0 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 0 (of expected -1) 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower760 (Follower): Setting last applied to 0 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from -1 to 0 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower760: Applying state for log index 0 data zero 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower760: applyState called: zero 09:09:29,639 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 317.0 μs 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): Commit index set to 1 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower760 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower760 (Follower): Setting last applied to 1 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from 0 to 1 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower760: Applying state for log index 1 data one 09:09:29,639 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower760: applyState called: one 09:09:29,639 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 138.6 μs 09:09:29,639 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands 09:09:29,639 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 73.49 μs 09:09:29,639 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 65.98 μs 09:09:29,639 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting to receive commands [WARN] [06/30/2025 09:09:29.639] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower760] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower760, initialSyncDone=true] [INFO] [pekkoDeadLetter][06/30/2025 09:09:29.639] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower760] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,639 AM [test-pekko.actor.default-dispatcher-5] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,718 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:09:29,718 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:09:29,839 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:09:29,839 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:09:29,839 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:09:29,881 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries: new entries applied - resuming follower follower761 09:09:29,958 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 09:09:29,958 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): Number of entries to be appended = 2 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): Append entry to log zero 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:29,959 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:09:29,959 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 1 commands after 981.8 ms 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): Append entry to log one 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): Log size is now 2 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): Commit index set to 1 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower761, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower761 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 0 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to 0 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower761: Applying state for log index 0 data zero 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower761: applyState called: zero 09:09:29,959 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: 1, nextIndex: 2 09:09:29,959 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader759: fakeSnapshot purging log to 0 for term 6 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 1 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from 0 to 1 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower761: Applying state for log index 1 data one 09:09:29,959 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:09:29,959 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower761: applyState called: one 09:09:29,960 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower761: awaiting more completions to resolve 2 deferred callback(s) 09:09:29,963 AM [follower761-writer-161] [DEBUG] EntryStoreCompleter - follower761: 1 completion(s) pending 09:09:29,963 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower761: running 1 completion(s) 09:09:29,963 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 4.629 ms 09:09:29,963 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 2 commands after 3.780 μs 09:09:29,963 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:29,964 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower761: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@f95ee61 09:09:29,964 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower761: awaiting more completions to resolve 1 deferred callback(s) 09:09:29,966 AM [follower761-writer-161] [DEBUG] EntryStoreCompleter - follower761: 1 completion(s) pending 09:09:29,966 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: completed 2 commands in 2.196 ms 09:09:29,966 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: waiting to receive commands 09:09:29,966 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower761: running 1 completion(s) 09:09:29,966 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:29,966 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower761: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@4a9761f7 [WARN] [06/30/2025 09:09:29.966] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=true] [INFO] [pekkoDeadLetter][06/30/2025 09:09:29.966] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:29,966 AM [test-pekko.actor.default-dispatcher-17] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,78 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:09:30,78 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 09:09:30,78 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower761: fakeSnapshot purging log to 0 for term 6 09:09:30,78 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower760: fakeSnapshot purging log to 0 for term 6 09:09:30,83 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries complete 09:09:30,83 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower760-collector762#-1719948030] 09:09:30,95 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower760] 09:09:30,95 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower760: terminating thread VirtualThread[#896,follower760-writer-160]/waiting 09:09:30,95 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 commands after 455.8 ms 09:09:30,95 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: completed 1 commands in 13.85 μs 09:09:30,95 AM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: journal writer stopped 09:09:30,95 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower760: journal closed 09:09:30,95 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower760: journal closed 09:09:30,95 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower760: stopped executor with 0 remaining tasks 09:09:30,95 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower760: stopped executor with 0 remaining tasks 09:09:30,95 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower761-collector763#200580877] 09:09:30,95 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower761] 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower761: terminating thread VirtualThread[#899,follower761-writer-161]/waiting 09:09:30,96 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 1 commands after 129.9 ms 09:09:30,96 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 11.12 μs 09:09:30,96 AM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: journal writer stopped 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower761: journal closed 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower761: journal closed 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks 09:09:30,96 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader759-collector764#-959801361] 09:09:30,96 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader759] 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader759: terminating thread VirtualThread[#900,leader759-writer-162]/waiting 09:09:30,96 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 commands after 458.4 ms 09:09:30,96 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: completed 1 commands in 11.24 μs 09:09:30,96 AM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: journal writer stopped 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader759: journal closed 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader759: journal closed 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader759: stopped executor with 0 remaining tasks 09:09:30,96 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader759: stopped executor with 0 remaining tasks 09:09:30,98 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaInstallSnapshot starting [WARN] [06/30/2025 09:09:30.100] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower769]: org.opendaylight.controller.cluster.common.actor.Monitor@48bd0fff [INFO] [pekkoDeadLetter][06/30/2025 09:09:30.100] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower769] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,100 AM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower769] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,100 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower769: Starting recovery with journal batch size 1000 09:09:30,100 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower769: started executor 09:09:30,100 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower769: started executor 09:09:30,101 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower769: metadata journal open: replayFrom=1 applyTo=0 09:09:30,101 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower769: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:30,101 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower769: journal open: applyTo=0 09:09:30,101 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: journal writer started 09:09:30,101 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands [WARN] [06/30/2025 09:09:30.102] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower770]: org.opendaylight.controller.cluster.common.actor.Monitor@5aa8fc92 [INFO] [pekkoDeadLetter][06/30/2025 09:09:30.102] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,102 AM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,103 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower770: Starting recovery with journal batch size 1000 09:09:30,103 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower770: started executor 09:09:30,103 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/follower769/TermInfo.properties does not exist 09:09:30,103 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower770: started executor 09:09:30,103 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/follower769/journal-v1-1.log 09:09:30,103 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/follower769/metadata-v1-1.log 09:09:30,103 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower770: metadata journal open: replayFrom=1 applyTo=0 09:09:30,103 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower769: no eligible files found 09:09:30,103 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower769: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:30,103 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower770: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:30,103 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower770: journal open: applyTo=0 09:09:30,103 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: journal writer started 09:09:30,103 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:09:30,104 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/follower770/TermInfo.properties does not exist 09:09:30,105 AM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader768] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,105 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/follower770/journal-v1-1.log 09:09:30,105 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/follower770/metadata-v1-1.log 09:09:30,105 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower770: no eligible files found 09:09:30,105 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower770: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:30,105 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 09:09:30,105 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: started executor 09:09:30,105 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: started executor 09:09:30,106 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=0 [WARN] [06/30/2025 09:09:30.106] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@334e5adc [INFO] [pekkoDeadLetter][06/30/2025 09:09:30.105] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader768] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:30,106 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:30,106 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=0 09:09:30,106 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: journal writer started 09:09:30,106 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:30,107 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties does not exist 09:09:30,107 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:30,107 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:30,107 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:30,107 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=0, votedFor=null 09:09:30,107 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:09:30,107 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower769: Local TermInfo store seeded with TermInfo{term=0} 09:09:30,107 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower769: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@778e847f 09:09:30,107 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:30,107 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from -1 to -1 09:09:30,108 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower769: journal recovery completedin 334.5 μs with journalIndex=1 09:09:30,110 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=0, votedFor=null 09:09:30,110 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower770: Local TermInfo store seeded with TermInfo{term=0} 09:09:30,110 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower770: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2ab89eb8 09:09:30,110 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to -1 09:09:30,111 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - follower770: journal recovery completedin 314.7 μs with journalIndex=1 09:09:30,111 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:09:30,111 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader768: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader768} 09:09:30,111 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@47f01748 09:09:30,111 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 09:09:30,112 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - leader768: journal recovery completedin 206.5 μs with journalIndex=1 09:09:30,628 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:09:30,628 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:30,628 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 09:09:30,628 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 09:09:30,632 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 09:09:30,632 AM [test-pekko.actor.default-dispatcher-17] [INFO] Candidate - leader768 (Candidate): Starting new election term 6 09:09:30,632 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} to peer follower769 09:09:30,632 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} to peer follower770 09:09:30,632 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:09:30,632 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:30,632 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 09:09:30,632 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:30,632 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower769 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:30,632 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:30,632 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 09:09:30,632 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:30,632 AM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower770 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:30,635 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=6, votedFor=null 09:09:30,636 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:30,637 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=null 09:09:30,637 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=6, votedFor=leader768 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Election: Leader has following peers: [follower769, follower770] 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:30,638 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:30,639 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:09:30,639 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Last sync leader does not match current leader leader768, need to catch up to -1 [WARN] [06/30/2025 09:09:30.639] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower769] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower769, initialSyncDone=false] 09:09:30,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: -1, nextIndex: 0 09:09:30,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:30,641 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=leader768 09:09:30,641 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:30,641 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Last sync leader does not match current leader leader768, need to catch up to -1 09:09:30,641 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/30/2025 09:09:30.641] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=false] 09:09:30,641 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:30,666 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:09:30,666 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=0 term=6 command=zero 09:09:30,666 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:30,666 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:09:30,666 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:30,666 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower769 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower770 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=1 term=6 command=one 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:09:30,667 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 2 commands after 560.6 ms 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Append entry to log zero 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Append entry to log zero 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Log size is now 1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:09:30,667 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: awaiting more completions to resolve 1 deferred callback(s) 09:09:30,667 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 565.8 ms 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:30,667 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 563.7 ms 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower769 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower770 09:09:30,667 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 0, nextIndex: 1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Log size is now 1 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:30,667 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:09:30,668 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 1 is present for follower follower770 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 0, nextIndex: 1 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 1 is present for follower follower769 09:09:30,668 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:30,671 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:09:30,671 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 3.878 ms 09:09:30,671 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@52d2d407 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Append entry to log one 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Log size is now 2 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:09:30,671 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 322.5 μs 09:09:30,671 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:09:30,672 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 1, nextIndex: 2 09:09:30,672 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:30,673 AM [follower770-writer-164] [DEBUG] EntryStoreCompleter - follower770: 1 completion(s) pending 09:09:30,673 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 5.462 ms 09:09:30,673 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: running 1 completion(s) 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@3970f70 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Append entry to log one 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Log size is now 2 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 09:09:30,673 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: awaiting more completions to resolve 1 deferred callback(s) 09:09:30,673 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 352.6 μs 09:09:30,673 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 1, nextIndex: 2 09:09:30,673 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:30,674 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:09:30,674 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:09:30,674 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 2.723 ms 09:09:30,674 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:30,674 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:30,674 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@5e949291 [WARN] [06/30/2025 09:09:30.674] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower769] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower769, initialSyncDone=true] 09:09:30,675 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 2 completion(s) pending 09:09:30,675 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 2 commands in 8.666 ms 09:09:30,676 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:30,676 AM [follower770-writer-164] [DEBUG] EntryStoreCompleter - follower770: 1 completion(s) pending 09:09:30,676 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 2.526 ms 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader768: running 2 completion(s) 09:09:30,676 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:09:30,676 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: running 1 completion(s) 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:30,676 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:30,676 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@45d7cd3b 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 0 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to 0 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Applying state for log index 0 data zero 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader768: applyState called: zero [WARN] [06/30/2025 09:09:30.676] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=true] 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 1 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 0 to 1 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Applying state for log index 1 data one 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader768: applyState called: one 09:09:30,676 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 598.2 μs 09:09:30,676 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader768: fakeSnapshot purging log to 0 for term 6 09:09:30,676 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 117.9 μs 09:09:30,676 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 4.860 μs 09:09:30,676 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 83.48 μs 09:09:30,676 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Commit index set to 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Commit index set to 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 0 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 0 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to 0 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from -1 to 0 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower770: Applying state for log index 0 data zero 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 0 data zero 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower770: applyState called: zero 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower769: applyState called: zero 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 0 to 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 1 data one 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower769: applyState called: one 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 0 to 1 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower770: Applying state for log index 1 data one 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower770: applyState called: one 09:09:30,879 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower770: fakeSnapshot purging log to 0 for term 6 09:09:30,879 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower769: fakeSnapshot purging log to 0 for term 6 09:09:30,879 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 203.7 ms 09:09:30,880 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 97.45 μs 09:09:30,880 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 205.3 ms 09:09:30,880 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting to receive commands 09:09:30,880 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 90.25 μs 09:09:30,880 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:30,999 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:30,999 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:31,118 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:31,239 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:31,359 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:31,425 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Persist data index=2 term=6 command=two 09:09:31,425 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:31,426 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 749.1 ms 09:09:31,426 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 09:09:31,426 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:31,426 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:09:31,426 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Persist data index=3 term=6 command=three 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Persist data index=4 term=6 command=four 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Persist data index=5 term=6 command=five 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log two 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Persist data index=6 term=6 command=six 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Log size is now 2 09:09:31,427 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:31,427 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:09:31,427 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 547.6 ms 09:09:31,428 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:09:31,428 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 2, nextIndex: 3 09:09:31,428 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:09:31,428 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 3 is present for follower follower769 09:09:31,428 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:09:31,429 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:09:31,429 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:09:31,429 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 3.593 ms 09:09:31,429 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 1.847 ms 09:09:31,429 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 4 commands after 4.970 μs 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:09:31,429 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@7a393789 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 4 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log three 09:09:31,429 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log four 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log five 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:09:31,430 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 304.1 μs 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log six 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Log size is now 6 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 09:09:31,430 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 4 deferred callback(s) 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 2 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 1 to 2 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Applying state for log index 2 data two 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader768: applyState called: two 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - leader768: fakeSnapshot purging log to 1 for term 6 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 6, nextIndex: 7 09:09:31,430 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1 09:09:31,431 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:09:31,431 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:09:31,431 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 1.077 ms 09:09:31,431 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 09:09:31,431 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 3 commands after 4.720 μs 09:09:31,431 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@61f46d3a 09:09:31,431 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 3 deferred callback(s) 09:09:31,435 AM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 4 completion(s) pending 09:09:31,435 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 3 completion(s) pending 09:09:31,435 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 4 commands in 5.305 ms 09:09:31,435 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 3 commands in 3.822 ms 09:09:31,435 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:31,435 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 4.060 μs 09:09:31,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader768: running 4 completion(s) 09:09:31,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:31,435 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 95.98 μs 09:09:31,435 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:31,435 AM [test-pekko.actor.default-dispatcher-5] [INFO] SnapshotManager - leader768: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=4, mandatoryTrim=false] 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 3 completion(s) 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false} 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@68b8a6ba 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false} 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@334b2180 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false} 09:09:31,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@6cc4743c 09:09:31,436 AM [test-pekko.actor.default-dispatcher-16] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) [ERROR] [06/30/2025 09:09:31.435] [test-pekko.actor.internal-dispatcher-4] [pekko://test/user/leader768] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:31,436 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#904,leader768-writer-165]/waiting 09:09:31,436 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 commands after 1.382 ms 09:09:31,436 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Commit index set to 2 09:09:31,436 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 8.390 μs 09:09:31,436 AM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: journal writer stopped 09:09:31,436 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 2 (of expected -1) 09:09:31,436 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 6 09:09:31,436 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: journal closed 09:09:31,436 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader768: journal closed 09:09:31,436 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:09:31,436 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:09:31.437] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@5acecbcf 09:09:31,438 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 2 09:09:31,438 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 1 to 2 09:09:31,438 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower769: Applying state for log index 2 data two 09:09:31,438 AM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower769: applyState called: two 09:09:31,438 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 09:09:31,438 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower769: fakeSnapshot purging log to 1 for term 6 09:09:31,438 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: started executor 09:09:31,438 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 3.295 ms 09:09:31,438 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 65.52 μs 09:09:31,438 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:31,438 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: started executor 09:09:31,439 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=2 09:09:31,440 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=7 replayFrom=1 fileEntries=0 09:09:31,440 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=2 09:09:31,440 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: journal writer started 09:09:31,440 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - leader768: restoring local TermInfo{term=6, votedFor=leader768} 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3c6ef8f8 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 09:09:31,442 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:09:31,443 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:09:31,443 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:09:31,443 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:09:31,443 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:09:31,443 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:09:31,444 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:09:31,444 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader768: journal recovery completedin 1.812 ms with journalIndex=1 09:09:31,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:31,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:31,728 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,728 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,728 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,728 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:31,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:31,899 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:31,899 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 09:09:31,899 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 09:09:31,903 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 09:09:31,903 AM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader768 (Candidate): Starting new election term 7 09:09:31,903 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} to peer follower769 09:09:31,903 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} to peer follower770 09:09:31,903 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 09:09:31,904 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:31,904 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 09:09:31,904 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:31,904 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 09:09:31,904 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 09:09:31,904 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 09:09:31,904 AM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower770 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:09:31,904 AM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower769 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:09:31,909 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=7, votedFor=null 09:09:31,909 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 6 09:09:31,910 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=7, votedFor=null 09:09:31,910 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 6, lastTerm: 6 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=7, votedFor=leader768 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader768 (Candidate): LastApplied index -1 is behind last index 6 - switching to PreLeader 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Election: Leader has following peers: [follower769, follower770] 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:09:31,912 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:09:31,913 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=7 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:31,913 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:31,913 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Replicate message: identifier: null, logIndex: 7, isSendImmediate: true 09:09:31,913 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:31,913 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:31,913 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: received 1 commands after 472.4 ms 09:09:31,913 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=7, votedFor=leader768 09:09:31,913 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:09:31,913 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 6, nextIndex: 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:09:31,913 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 7 is present for follower follower769 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Sending AppendEntries to follower follower769: AppendEntries{term=7, leaderId=leader768, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 7 is present for follower follower769 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader768, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): Log size is now 6 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower769, logLastIndex=7, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:31,914 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 7, nextIndex: 8 09:09:31,914 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 476.3 ms 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:09:31,914 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:31,915 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:09:31,915 AM [leader768-writer-166] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:09:31,915 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:09:31,915 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 2.564 ms 09:09:31,915 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:31,915 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:31,916 AM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader768: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=7, lastTerm=7, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=8, mandatoryTrim=false] 09:09:31,916 AM [test-pekko.actor.default-dispatcher-12] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:31,916 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#905,leader768-writer-166]/waiting 09:09:31,916 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: received 1 commands after 914.7 μs [ERROR] [06/30/2025 09:09:31.916] [test-pekko.actor.internal-dispatcher-10] [pekko://test/user/leader768] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:31,916 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 32.89 μs 09:09:31,916 AM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: journal writer stopped 09:09:31,916 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:09:31,917 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:09:31,917 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 2.117 ms 09:09:31,917 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=false} 09:09:31,917 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:31,917 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@71fd057a 09:09:31,917 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader768: journal closed 09:09:31,917 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader768: journal closed 09:09:31,917 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:09:31,917 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:09:31.917] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@18edbe1e 09:09:31,918 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 09:09:31,918 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: started executor 09:09:31,918 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: started executor 09:09:31,919 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=2 09:09:31,921 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=8 replayFrom=1 fileEntries=0 09:09:31,921 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=2 09:09:31,921 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: journal writer started 09:09:31,921 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader768: restoring local TermInfo{term=7, votedFor=leader768} 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@64f95ef3 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 09:09:31,922 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b] 09:09:31,923 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader768: journal recovery completedin 1.191 ms with journalIndex=1 09:09:31,929 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:31,929 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:31,929 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:31,929 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,130 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,130 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,130 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,130 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,231 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,231 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,231 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,231 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,331 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,331 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,331 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,332 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,429 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:32,429 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 09:09:32,429 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 09:09:32,432 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties12584785018674006531.tmp 09:09:32,432 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,432 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,432 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,432 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,432 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=8, votedFor=leader768 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader768 (Candidate): Starting new election term 8 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} to peer follower769 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} to peer follower770 09:09:32,433 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 8 09:09:32,433 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 09:09:32,433 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 09:09:32,433 AM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower769 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 09:09:32,433 AM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower770 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 09:09:32,436 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=8, votedFor=null 09:09:32,436 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 6 09:09:32,437 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=8, votedFor=null 09:09:32,437 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 7, lastTerm: 7 09:09:32,439 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=8, votedFor=leader768 09:09:32,439 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 09:09:32,439 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=8, voteGranted=true}, my term is 8 09:09:32,439 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=8, voteGranted=true}, current voteCount: 1 09:09:32,439 AM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader768 (Candidate): LastApplied index -1 is behind last index 7 - switching to PreLeader 09:09:32,439 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (PreLeader): Election: Leader has following peers: [follower769, follower770] 09:09:32,439 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:32,439 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:32,440 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=8 term=8 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:32,440 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:32,440 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (PreLeader): Replicate message: identifier: null, logIndex: 8, isSendImmediate: true 09:09:32,440 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:32,440 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:32,440 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 commands after 518.8 ms 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=8, votedFor=leader768 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 7, nextIndex: 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 8 is present for follower follower769 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Sending AppendEntries to follower follower769: AppendEntries{term=8, leaderId=leader768, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:32,440 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 8 is present for follower follower769 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=8, leaderId=leader768, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Log size is now 7 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=8, success=true, followerId=follower769, logLastIndex=8, logLastTerm=8, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 09:09:32,441 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 09:09:32,441 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 524.3 ms 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 8, nextIndex: 9 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:32,441 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:32,442 AM [leader768-writer-167] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 09:09:32,442 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 1.817 ms 09:09:32,442 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Applying to log - commitIndex: 8, lastAppliedIndex: -1 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 0 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to 0 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 0 data zero 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: zero 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 1 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 0 to 1 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 1 data one 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: one 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 2 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 1 to 2 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 2 data two 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: two 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 3 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 2 to 3 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 3 data three 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: three 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 4 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 3 to 4 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 4 data four 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: four 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 5 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 4 to 5 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 5 data five 09:09:32,442 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: five 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 6 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 5 to 6 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 6 data six 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: six 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 7 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 6 to 7 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 8 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 7 to 8 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:32,443 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 commands after 1.029 ms 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PreLeader - leader768 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 8, lastIndex: 8 09:09:32,443 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 69.01 μs 09:09:32,443 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting to receive commands 09:09:32,443 AM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 09:09:32,443 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 2.018 ms 09:09:32,443 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=false} 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@71e3908a 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (Leader): Election: Leader has following peers: [follower769, follower770] 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): Commit index set to 8 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 2, lastIndex(): 8 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 3 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 2 to 3 09:09:32,443 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader768 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 8 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 3 data three 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower769: applyState called: three 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 4 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 3 to 4 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 4 data four 09:09:32,443 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower769: applyState called: four 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 5 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 4 to 5 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 5 data five 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower769: applyState called: five 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 6 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 5 to 6 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 6 data six 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower769: applyState called: six 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 7 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 6 to 7 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 8 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 7 to 8 09:09:32,444 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower769: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:32,444 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 763.3 μs 09:09:32,444 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 100.0 μs 09:09:32,444 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting to receive commands 09:09:32,533 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,533 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,533 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,533 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,559 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:32,633 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,634 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,634 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,634 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,679 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:32,734 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,734 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,734 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,734 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,798 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:32,835 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,835 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,835 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,835 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:32,918 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:32,935 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:32,935 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:32,935 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:32,935 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,36 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,36 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,36 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,36 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,38 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,137 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,137 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,137 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,137 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,159 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,237 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,237 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,238 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,238 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,278 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,338 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,338 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,338 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,338 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,399 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,439 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,439 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,439 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,439 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,519 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,539 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,539 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,539 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,539 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,638 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,640 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,640 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,640 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,640 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,741 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,741 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,741 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,741 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,758 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,842 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,842 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,842 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,842 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,879 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:33,943 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:33,943 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:33,943 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:33,943 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:33,999 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,43 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,43 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,43 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,43 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,119 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,144 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,144 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,144 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,144 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,238 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,245 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,245 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,245 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,245 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,345 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,345 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,345 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,346 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,358 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,446 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,446 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,446 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,446 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,479 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,546 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,546 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,546 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,546 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,599 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,647 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,647 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,647 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,647 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,719 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,748 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,748 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,748 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,748 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,839 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:34,848 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,849 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,849 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,849 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,949 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:34,949 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:34,949 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:34,949 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:34,958 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,50 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,50 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,50 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,50 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,78 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,150 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,151 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,151 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,151 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,198 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,251 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,251 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,251 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,251 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,319 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,352 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,352 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,352 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,352 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,439 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,452 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,452 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,452 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,452 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,553 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,553 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,553 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,553 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,559 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,654 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,654 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,654 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,654 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,679 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,755 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,755 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,755 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,755 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,798 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,855 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,855 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,855 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,855 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:35,919 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:35,956 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:35,956 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:35,956 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:35,956 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:36,39 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:36,56 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:36,56 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:36,56 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:36,56 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:36,157 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:36,157 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:36,157 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:36,157 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:36,159 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:36,258 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:36,258 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:36,258 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:36,258 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:36,278 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:36,358 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/TermInfo.properties 09:09:36,358 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/journal-v1-1.log 09:09:36,358 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3561863098555432249/leader768/metadata-v1-1.log 09:09:36,358 AM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 09:09:36,399 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 09:09:36,460 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower769-collector771#1480728702] 09:09:36,471 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower769] 09:09:36,471 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower769: terminating thread VirtualThread[#902,follower769-writer-163]/waiting 09:09:36,471 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 commands after 4.027 s 09:09:36,471 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: completed 1 commands in 14.18 μs 09:09:36,471 AM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: journal writer stopped 09:09:36,471 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower769: journal closed 09:09:36,471 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower769: journal closed 09:09:36,471 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower769: stopped executor with 0 remaining tasks 09:09:36,471 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower769: stopped executor with 0 remaining tasks 09:09:36,472 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower770-collector772#863016546] 09:09:36,472 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower770] 09:09:36,472 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower770: terminating thread VirtualThread[#903,follower770-writer-164]/waiting 09:09:36,472 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 commands after 5.593 s 09:09:36,472 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 11.74 μs 09:09:36,472 AM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: journal writer stopped 09:09:36,472 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower770: journal closed 09:09:36,472 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower770: journal closed 09:09:36,472 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks 09:09:36,472 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks 09:09:36,473 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader768-collector773#695307095] 09:09:36,473 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader768] 09:09:36,473 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#907,leader768-writer-167]/waiting 09:09:36,473 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 commands after 4.030 s 09:09:36,473 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: completed 1 commands in 9.091 μs 09:09:36,473 AM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: journal writer stopped 09:09:36,473 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader768: journal closed 09:09:36,473 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader768: journal closed 09:09:36,473 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:09:36,473 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 09:09:36,476 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaAppendEntries starting [WARN] [06/30/2025 09:09:36.478] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower778]: org.opendaylight.controller.cluster.common.actor.Monitor@4cd508a2 09:09:36,478 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower778: Starting recovery with journal batch size 1000 09:09:36,479 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower778: started executor 09:09:36,479 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower778: started executor 09:09:36,479 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower778: metadata journal open: replayFrom=1 applyTo=0 09:09:36,480 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower778: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:36,480 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower778: journal open: applyTo=0 09:09:36,480 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: journal writer started 09:09:36,480 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:36,481 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/follower778/TermInfo.properties does not exist 09:09:36,481 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/follower778/journal-v1-1.log 09:09:36,481 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/follower778/metadata-v1-1.log 09:09:36,481 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: no eligible files found 09:09:36,481 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower778: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:09:36.481] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower779]: org.opendaylight.controller.cluster.common.actor.Monitor@7341424d 09:09:36,481 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower779: Starting recovery with journal batch size 1000 09:09:36,481 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower779: started executor 09:09:36,481 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower779: started executor 09:09:36,482 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower779: metadata journal open: replayFrom=1 applyTo=0 09:09:36,482 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower779: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:36,482 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower779: journal open: applyTo=0 09:09:36,482 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: journal writer started 09:09:36,482 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:09:36,483 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/follower779/TermInfo.properties does not exist 09:09:36,483 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/follower779/journal-v1-1.log 09:09:36,483 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/follower779/metadata-v1-1.log 09:09:36,483 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower779: no eligible files found 09:09:36,483 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower779: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:09:36.483] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader777]: org.opendaylight.controller.cluster.common.actor.Monitor@31dc4b4f 09:09:36,484 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - leader777: Starting recovery with journal batch size 1000 09:09:36,484 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: started executor 09:09:36,484 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: started executor 09:09:36,484 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: metadata journal open: replayFrom=1 applyTo=0 09:09:36,484 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:36,484 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader777: journal open: applyTo=0 09:09:36,484 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: journal writer started 09:09:36,484 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:36,485 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties does not exist 09:09:36,486 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:36,486 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:36,486 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:36,486 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 09:09:36,486 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader777: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:36,488 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=0, votedFor=null 09:09:36,488 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower778: Local TermInfo store seeded with TermInfo{term=0} 09:09:36,488 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower778: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7c139c9a 09:09:36,488 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from -1 to -1 09:09:36,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower778: journal recovery completedin 163.2 μs with journalIndex=1 09:09:36,491 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 09:09:36,491 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader777: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader777} 09:09:36,491 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3f5e70da 09:09:36,491 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=0, votedFor=null 09:09:36,491 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to -1 09:09:36,491 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower779: Local TermInfo store seeded with TermInfo{term=0} 09:09:36,491 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower779: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@27492f50 09:09:36,491 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to -1 09:09:36,491 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - leader777: journal recovery completedin 144.6 μs with journalIndex=1 09:09:36,491 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower779: journal recovery completedin 206.2 μs with journalIndex=1 09:09:36,998 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:09:36,998 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader777 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:36,998 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader777 (Follower): Received ElectionTimeout - switching to Candidate 09:09:36,998 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader777 (Candidate): Election: Candidate has following voting peers: [follower779, follower778] 09:09:37,2 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 09:09:37,2 AM [test-pekko.actor.default-dispatcher-17] [INFO] Candidate - leader777 (Candidate): Starting new election term 6 09:09:37,2 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} to peer follower779 09:09:37,2 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} to peer follower778 09:09:37,2 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader777 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:09:37,3 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:37,3 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:37,3 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: [] 09:09:37,3 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:37,3 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): Cluster state: [] 09:09:37,3 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower779 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:37,3 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:37,3 AM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower778 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:37,6 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=null 09:09:37,6 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:37,7 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=6, votedFor=null 09:09:37,7 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): In requestVote: RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:37,9 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=leader777 09:09:37,9 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Election: Leader has following peers: [follower779, follower778] 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader777 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:09:37,10 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower779: Last sync leader does not match current leader leader777, need to catch up to -1 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: -1, nextIndex: 0 09:09:37,10 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:37.010] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=false] 09:09:37,10 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=6, votedFor=leader777 09:09:37,10 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:37,11 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Last sync leader does not match current leader leader777, need to catch up to -1 09:09:37,11 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: -1, nextIndex: 0 09:09:37,11 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:37.011] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower778] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower778, initialSyncDone=false] 09:09:37,44 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader777: Persist data index=0 term=6 command=zero 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower779 09:09:37,45 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 560.2 ms 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower778 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader777: Persist data index=1 term=6 command=one 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower779 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower778 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Append entry to log zero 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Log size is now 1 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Append entry to log zero 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Log size is now 1 09:09:37,45 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 563.3 ms 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,45 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:09:37,45 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:09:37,46 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,46 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,46 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 565.7 ms 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 0, nextIndex: 1 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 1 is present for follower follower778 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 0, nextIndex: 1 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 1 is present for follower follower779 09:09:37,46 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:37,51 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:09:37,51 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 6.022 ms 09:09:37,51 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 3.770 μs 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 0 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to 0 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 0 data zero 09:09:37,51 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader777: applyState called: zero 09:09:37,52 AM [follower779-writer-169] [DEBUG] EntryStoreCompleter - follower779: 1 completion(s) pending 09:09:37,52 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:09:37,52 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 6.626 ms 09:09:37,52 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:09:37,52 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 6.376 ms 09:09:37,52 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: running 1 completion(s) 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@69c94a27 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@327f2f38 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Append entry to log one 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Append entry to log one 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Log size is now 2 09:09:37,52 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Log size is now 2 09:09:37,52 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,53 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,53 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:09:37,53 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 459.7 μs 09:09:37,53 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 09:09:37,53 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,53 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 1, nextIndex: 2 09:09:37,53 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 563.7 μs 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 1, nextIndex: 2 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:37,53 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:09:37,53 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.457 ms 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:09:37,53 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 3.200 μs 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 1 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 0 to 1 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Applying state for log index 1 data one 09:09:37,53 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader777: applyState called: one 09:09:37,54 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader777: fakeSnapshot purging log to 0 for term 6 09:09:37,54 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 177.9 μs 09:09:37,54 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 3.170 μs 09:09:37,54 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 51.39 μs 09:09:37,54 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,54 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:09:37,54 AM [follower779-writer-169] [DEBUG] EntryStoreCompleter - follower779: 1 completion(s) pending 09:09:37,54 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 1.773 ms 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,54 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 1.916 ms 09:09:37,55 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@1ebc7943 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: running 1 completion(s) 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower779: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@4c4d107a 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Commit index set to 0 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 0 (of expected -1) 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 0 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from -1 to 0 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower778: Applying state for log index 0 data zero 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower778: applyState called: zero 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Commit index set to 0 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 0 (of expected -1) 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 0 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to 0 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower779: Applying state for log index 0 data zero 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower779: applyState called: zero 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 326.9 μs 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 102.9 μs 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): Commit index set to 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 1 09:09:37,55 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 497.9 μs 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 0 to 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower778: Applying state for log index 1 data one 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower778: applyState called: one 09:09:37,55 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 81.36 μs 09:09:37,55 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - follower778: fakeSnapshot purging log to 0 for term 6 09:09:37,55 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 177.1 μs 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 72.06 μs 09:09:37,55 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): Commit index set to 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 0 to 1 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower779: Applying state for log index 1 data one 09:09:37,55 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower779: applyState called: one [WARN] [06/30/2025 09:09:37.055] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower778] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower778, initialSyncDone=true] 09:09:37,56 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower779: fakeSnapshot purging log to 0 for term 6 09:09:37,56 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 396.3 μs 09:09:37,56 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 62.42 μs 09:09:37,56 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting to receive commands [WARN] [06/30/2025 09:09:37.056] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=true] 09:09:37,248 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:09:37,248 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader777: Persist data index=2 term=6 command=two 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:09:37,297 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 243.3 ms 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower778 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Append entry to log two 09:09:37,297 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:37,297 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 242.1 ms 09:09:37,298 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Log size is now 2 09:09:37,298 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,298 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 09:09:37,298 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,298 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 2, nextIndex: 3 09:09:37,298 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:37,300 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:09:37,300 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.615 ms 09:09:37,300 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:09:37,300 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:09:37,300 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 2.230 ms 09:09:37,300 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 2 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 1 to 2 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 2 data two 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader777: applyState called: two 09:09:37,300 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:09:37,300 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 09:09:37,300 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@6c828f65 09:09:37,300 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader777: fakeSnapshot purging log to 1 for term 6 09:09:37,300 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 388.5 μs 09:09:37,300 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 99.44 μs 09:09:37,300 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,368 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:09:37,368 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}]} 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}]} 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): Commit index set to 2 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 2 (of expected -1) 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 2 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 1 to 2 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower778: Applying state for log index 2 data two 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower778: applyState called: two 09:09:37,489 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower778: fakeSnapshot purging log to 1 for term 6 09:09:37,489 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 189.4 ms 09:09:37,489 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 95.84 μs 09:09:37,489 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,498 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader777: Persist data index=3 term=6 command=three 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:37,499 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 198.4 ms 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 3 is present for follower follower778 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]} 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]} 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Append entry to log three 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Log size is now 2 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 2 (of expected -1) 09:09:37,499 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,499 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 10.07 ms 09:09:37,500 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 3, nextIndex: 4 09:09:37,500 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:09:37,501 AM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:09:37,501 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.490 ms 09:09:37,501 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,501 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:09:37,501 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:37,501 AM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader777: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false] 09:09:37,502 AM [test-pekko.actor.default-dispatcher-5] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [ERROR] [06/30/2025 09:09:37.502] [test-pekko.actor.internal-dispatcher-3] [pekko://test/user/leader777] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:37,502 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - leader777: terminating thread VirtualThread[#910,leader777-writer-170]/waiting 09:09:37,502 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:09:37,503 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 3.056 ms 09:09:37,503 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,503 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 commands after 1.348 ms 09:09:37,503 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:09:37,503 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 12.83 μs 09:09:37,503 AM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: journal writer stopped 09:09:37,503 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 09:09:37,503 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@4bdf84d9 09:09:37,503 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: journal closed 09:09:37,503 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader777: journal closed 09:09:37,503 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:09:37,503 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:09:37,504 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - leader777: Starting recovery with journal batch size 1000 09:09:37,504 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: started executor 09:09:37,504 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: started executor [WARN] [06/30/2025 09:09:37.505] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader777]: org.opendaylight.controller.cluster.common.actor.Monitor@175e7ad3 09:09:37,505 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: metadata journal open: replayFrom=1 applyTo=2 09:09:37,506 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: journal open: firstIndex=1 lastIndex=4 replayFrom=1 fileEntries=0 09:09:37,506 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader777: journal open: applyTo=2 09:09:37,506 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: journal writer started 09:09:37,506 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader777: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRecovery - leader777: restoring local TermInfo{term=6, votedFor=leader777} 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@619b39e9 09:09:37,508 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to -1 09:09:37,509 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:09:37,509 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:09:37,509 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:09:37,510 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:09:37,510 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader777: journal recovery completedin 1.391 ms with journalIndex=1 09:09:37,599 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:37,599 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:37,599 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:37,599 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:37,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:37,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:37,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:37,700 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:37,800 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:37,800 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:37,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:37,801 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:37,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:37,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:37,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:37,901 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:37,959 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader777 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:37,959 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader777 (Follower): Received ElectionTimeout - switching to Candidate 09:09:37,959 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader777 (Candidate): Election: Candidate has following voting peers: [follower779, follower778] 09:09:37,962 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=7, votedFor=leader777 09:09:37,962 AM [test-pekko.actor.default-dispatcher-17] [INFO] Candidate - leader777 (Candidate): Starting new election term 7 09:09:37,962 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} to peer follower779 09:09:37,962 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} to peer follower778 09:09:37,962 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader777 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 09:09:37,962 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:37,962 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: [] 09:09:37,962 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} with higher term appears reachable, updating term. 09:09:37,962 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:37,962 AM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower779 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:09:37,962 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): Cluster state: [] 09:09:37,962 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} with higher term appears reachable, updating term. 09:09:37,963 AM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower778 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:09:37,968 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=7, votedFor=null 09:09:37,968 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): In requestVote: RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 3, lastTerm: 6 09:09:37,969 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=7, votedFor=null 09:09:37,970 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 6 09:09:37,971 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=7, votedFor=leader777 09:09:37,971 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader777 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader777 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader777 (Candidate): LastApplied index -1 is behind last index 3 - switching to PreLeader 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Election: Leader has following peers: [follower779, follower778] 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader777 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader777: Persist data index=4 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:37,972 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 09:09:37,972 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:37,972 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 commands after 466.0 ms 09:09:37,973 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=7, votedFor=leader777 09:09:37,973 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Replicate message: identifier: null, logIndex: 4, isSendImmediate: true 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 3, nextIndex: 4 09:09:37,973 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:37,973 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): sendAppendEntries: 4 is present for follower follower778 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Sending AppendEntries to follower follower778: AppendEntries{term=7, leaderId=leader777, prevLogIndex=3, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader777, prevLogIndex=3, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): sendAppendEntries: 4 is present for follower follower778 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Log size is now 3 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower778, logLastIndex=4, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 09:09:37,974 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 471.6 ms 09:09:37,974 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 4, nextIndex: 5 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:37,974 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:09:37,975 AM [leader777-writer-171] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 09:09:37,975 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 2.831 ms 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 09:09:37,975 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (PreLeader): Applying to log - commitIndex: 4, lastAppliedIndex: -1 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 0 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to 0 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 0 data zero 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader777: applyState called: zero 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 1 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 0 to 1 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 1 data one 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader777: applyState called: one 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 2 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 1 to 2 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 2 data two 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader777: applyState called: two 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 3 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 2 to 3 09:09:37,975 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 3 data three 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader777: applyState called: three 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 4 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 3 to 4 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader777: Applying state for log index 4 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PreLeader - leader777 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 4, lastIndex: 4 09:09:37,976 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 commands after 461.1 μs 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Election: Leader has following peers: [follower779, follower778] 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:09:37,976 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 90.59 μs 09:09:37,976 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting to receive commands 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1 09:09:37,976 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader777 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 7 09:09:37,976 AM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 09:09:37,976 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 09:09:37,976 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 2.149 ms 09:09:37,976 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=false} 09:09:37,976 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:37,976 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@2c151d3b 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): Commit index set to 4 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 3 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 2 to 3 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower778: Applying state for log index 3 data three 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower778: applyState called: three 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 4 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 3 to 4 09:09:37,977 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower778: Applying state for log index 4 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:37,977 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 344.8 μs 09:09:37,977 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 68.52 μs 09:09:37,977 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting to receive commands 09:09:38,1 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,2 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,89 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,102 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,202 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,203 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,208 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,303 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,328 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,403 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,449 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,504 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,569 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,604 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,605 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,688 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,705 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,806 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,806 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,806 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,806 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,808 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:38,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:38,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:38,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:38,906 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:38,928 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,7 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,7 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,7 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,7 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,49 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,107 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,169 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,208 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,208 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,208 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,208 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,288 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,308 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,409 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,409 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,409 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,409 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,409 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,509 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,529 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,610 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,649 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,710 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,768 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,811 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:39,888 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:39,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:39,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:39,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:39,911 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,9 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,11 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,11 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,12 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,12 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,112 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,112 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,112 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,112 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,128 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,213 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,248 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,313 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,313 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,313 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,313 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,369 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,414 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,488 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,514 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,514 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,514 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,514 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,608 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,615 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,615 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,615 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,615 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,715 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,715 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,715 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,715 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,728 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,816 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,816 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,816 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,816 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,848 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:40,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:40,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:40,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:40,917 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:40,968 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,18 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,89 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,118 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,118 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,118 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,118 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,209 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,219 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,219 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,219 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,219 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,319 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,319 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,319 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,319 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,328 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,420 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,420 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,420 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,420 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,448 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,520 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,520 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,520 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,520 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,569 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,621 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,621 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,621 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,621 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,688 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,721 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,721 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,721 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,721 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,808 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:41,822 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,822 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,822 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,822 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,923 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:41,923 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:41,923 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:41,923 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:41,929 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:42,24 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:42,24 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:42,24 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:42,24 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:42,48 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:42,125 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:42,125 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:42,125 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:42,125 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:42,168 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:42,225 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:42,225 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:42,225 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:42,225 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:42,289 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:42,326 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:42,326 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:42,326 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:42,326 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:42,409 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:42,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/TermInfo.properties 09:09:42,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/journal-v1-1.log 09:09:42,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest13245042893986975082/leader777/metadata-v1-1.log 09:09:42,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 09:09:42,527 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower778-collector780#-809523894] 09:09:42,528 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 09:09:42,539 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower778] 09:09:42,539 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower778: terminating thread VirtualThread[#908,follower778-writer-168]/waiting 09:09:42,539 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 commands after 4.562 s 09:09:42,539 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: completed 1 commands in 15.52 μs 09:09:42,539 AM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: journal writer stopped 09:09:42,539 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower778: journal closed 09:09:42,539 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower778: journal closed 09:09:42,539 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: stopped executor with 0 remaining tasks 09:09:42,539 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: stopped executor with 0 remaining tasks 09:09:42,540 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower779-collector781#-1195279725] 09:09:42,540 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower779] 09:09:42,540 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower779: terminating thread VirtualThread[#909,follower779-writer-169]/waiting 09:09:42,540 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 commands after 5.484 s 09:09:42,540 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 11.75 μs 09:09:42,540 AM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: journal writer stopped 09:09:42,540 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower779: journal closed 09:09:42,540 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower779: journal closed 09:09:42,540 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks 09:09:42,540 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks 09:09:42,540 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader777-collector782#-32252543] 09:09:42,541 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader777] 09:09:42,541 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader777: terminating thread VirtualThread[#912,leader777-writer-171]/waiting 09:09:42,541 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 commands after 4.565 s 09:09:42,541 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: completed 1 commands in 11.57 μs 09:09:42,541 AM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: journal writer stopped 09:09:42,541 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader777: journal closed 09:09:42,541 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader777: journal closed 09:09:42,541 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:09:42,541 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 09:09:42,544 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderInstallsSnapshotWithRestartedFollowerDuringSnapshotInstallation starting [WARN] [06/30/2025 09:09:42.545] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower787]: org.opendaylight.controller.cluster.common.actor.Monitor@6d78e1ef 09:09:42,546 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - follower787: Starting recovery with journal batch size 1000 09:09:42,546 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: started executor 09:09:42,547 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: started executor 09:09:42,547 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower787: metadata journal open: replayFrom=1 applyTo=0 09:09:42,548 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower787: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:42,548 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower787: journal open: applyTo=0 09:09:42,548 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: journal writer started 09:09:42,548 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:42,549 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/follower787/TermInfo.properties does not exist 09:09:42,549 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/follower787/journal-v1-1.log 09:09:42,549 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/follower787/metadata-v1-1.log 09:09:42,549 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: no eligible files found 09:09:42,549 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower787: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:42,555 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=0, votedFor=null 09:09:42,555 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower787: Local TermInfo store seeded with TermInfo{term=0} 09:09:42,555 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower787: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@41ef47a6 09:09:42,555 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from -1 to -1 09:09:42,555 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower787: journal recovery completedin 165.7 μs with journalIndex=1 [WARN] [06/30/2025 09:09:42.647] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower788]: org.opendaylight.controller.cluster.common.actor.Monitor@3b59a6c1 09:09:42,648 AM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower788: Starting recovery with journal batch size 1000 09:09:42,649 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: started executor 09:09:42,649 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: started executor 09:09:42,649 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower788: metadata journal open: replayFrom=1 applyTo=0 09:09:42,650 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower788: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:42,650 AM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower788: journal open: applyTo=0 09:09:42,650 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: journal writer started 09:09:42,650 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:09:42,651 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/follower788/TermInfo.properties does not exist 09:09:42,651 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/follower788/journal-v1-1.log 09:09:42,651 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/follower788/metadata-v1-1.log 09:09:42,651 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: no eligible files found 09:09:42,651 AM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower788: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:09:42.652] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@7d866d18 09:09:42,653 AM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 09:09:42,653 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: started executor 09:09:42,653 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: started executor 09:09:42,654 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=0 09:09:42,654 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:42,654 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=0 09:09:42,655 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: journal writer started 09:09:42,655 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:42,655 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=0, votedFor=null 09:09:42,655 AM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower788: Local TermInfo store seeded with TermInfo{term=0} 09:09:42,655 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower788: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@18198305 09:09:42,655 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to -1 09:09:42,656 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] JournalRecovery - follower788: journal recovery completedin 248.8 μs with journalIndex=1 09:09:42,656 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties does not exist 09:09:42,656 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:42,656 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:42,656 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:42,656 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:09:42,656 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:42,659 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:09:42,659 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader786: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader786} 09:09:42,659 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2389e3ea 09:09:42,659 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 09:09:42,659 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - leader786: journal recovery completedin 243.8 μs with journalIndex=1 09:09:43,129 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:09:43,129 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:43,129 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 09:09:43,129 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [INFO] Candidate - leader786 (Candidate): Starting new election term 6 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} to peer follower788 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} to peer follower787 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): Cluster state: [] 09:09:43,133 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:43,134 AM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower788 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:43,134 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:43,134 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 09:09:43,134 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:43,134 AM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower787 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:43,137 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=null 09:09:43,137 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): In requestVote: RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:43,139 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=6, votedFor=null 09:09:43,139 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=leader786 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): Election: Leader has following peers: [follower788, follower787] 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:09:43,140 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Last sync leader does not match current leader leader786, need to catch up to -1 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: -1, nextIndex: 0 09:09:43,140 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:43.141] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=false] 09:09:43,142 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=6, votedFor=leader786 09:09:43,142 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:43,142 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Last sync leader does not match current leader leader786, need to catch up to -1 09:09:43,142 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: -1, nextIndex: 0 09:09:43,142 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:43.142] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower787] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower787, initialSyncDone=false] 09:09:43,164 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:09:43,164 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=0 term=6 command=zero 09:09:43,164 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:43,164 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:09:43,164 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:43,164 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 509.6 ms 09:09:43,164 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower788 09:09:43,164 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Append entry to log zero 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Log size is now 1 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:09:43,165 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 514.7 ms 09:09:43,165 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: awaiting more completions to resolve 1 deferred callback(s) 09:09:43,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:43,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower787 09:09:43,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Append entry to log zero 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Log size is now 1 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:09:43,165 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:09:43,165 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 617.3 ms 09:09:43,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 0, nextIndex: 1 09:09:43,165 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 0, nextIndex: 1 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=1 term=6 command=one 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 1 is present for follower follower788 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 1 is present for follower follower787 09:09:43,166 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:43,170 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:09:43,170 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 5.287 ms 09:09:43,170 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 4.450 μs 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 0 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to 0 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Applying state for log index 0 data zero 09:09:43,170 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - leader786: applyState called: zero 09:09:43,171 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:09:43,171 AM [follower788-writer-173] [DEBUG] EntryStoreCompleter - follower788: 1 completion(s) pending 09:09:43,171 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 5.638 ms 09:09:43,171 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 6.099 ms 09:09:43,171 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:43,171 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@3c5c1136 09:09:43,171 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: running 1 completion(s) 09:09:43,171 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:43,171 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@7f37c466 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Append entry to log one 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Log size is now 2 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:09:43,171 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 285.3 μs 09:09:43,171 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:09:43,172 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 1, nextIndex: 2 09:09:43,172 AM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Append entry to log one 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Log size is now 2 09:09:43,172 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 760.2 μs 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 09:09:43,172 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: awaiting more completions to resolve 1 deferred callback(s) 09:09:43,172 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 1, nextIndex: 2 09:09:43,172 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:09:43,172 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 2.473 ms 09:09:43,172 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 3.320 μs 09:09:43,172 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:43,172 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:09:43,172 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:43,173 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 299.8 μs 09:09:43,173 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:43,173 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:09:43,173 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 1 09:09:43,173 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 0 to 1 09:09:43,173 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader786: Applying state for log index 1 data one 09:09:43,173 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader786: applyState called: one 09:09:43,173 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 361.8 μs 09:09:43,173 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 75.11 μs 09:09:43,173 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:43,173 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader786: fakeSnapshot purging log to 0 for term 6 09:09:43,173 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:09:43,173 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 1.879 ms 09:09:43,173 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:43,173 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:09:43,173 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:43,173 AM [follower788-writer-173] [DEBUG] EntryStoreCompleter - follower788: 1 completion(s) pending 09:09:43,173 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@70eef6ce 09:09:43,173 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: running 1 completion(s) 09:09:43,173 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:43,173 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 1.534 ms 09:09:43,173 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower788: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@6db1714b 09:09:43,173 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:09:43,173 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Commit index set to 0 09:09:43,173 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Commit index set to 0 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 0 (of expected -1) 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 0 (of expected -1) 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 0 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to 0 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower788: Applying state for log index 0 data zero 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 0 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower788: applyState called: zero 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from -1 to 0 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 0 data zero 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: zero 09:09:43,174 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 215.7 μs 09:09:43,174 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 48.48 μs 09:09:43,174 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 424.8 μs 09:09:43,174 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Commit index set to 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,174 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 56.66 μs 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:09:43,174 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 0 to 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 1 data one 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: one 09:09:43,174 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower787: fakeSnapshot purging log to 0 for term 6 09:09:43,174 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 120.9 μs 09:09:43,174 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 41.60 μs 09:09:43,174 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands [WARN] [06/30/2025 09:09:43.174] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=true] [WARN] [06/30/2025 09:09:43.174] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower787] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower787, initialSyncDone=true] 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): Commit index set to 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 0 to 1 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower788: Applying state for log index 1 data one 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower788: applyState called: one 09:09:43,174 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 617.6 μs 09:09:43,174 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 44.08 μs 09:09:43,174 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting to receive commands 09:09:43,174 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower788: fakeSnapshot purging log to 0 for term 6 09:09:43,378 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,378 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,416 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - follower788: terminating thread VirtualThread[#915,follower788-writer-173]/waiting 09:09:43,416 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 commands after 241.9 ms 09:09:43,416 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 11.93 μs 09:09:43,416 AM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: journal writer stopped 09:09:43,417 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower788: journal closed 09:09:43,417 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower788: journal closed 09:09:43,417 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks 09:09:43,417 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:09:43.498] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:09:43,498 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,619 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) [WARN] [06/30/2025 09:09:43.619] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [06/30/2025 09:09:43.738] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:09:43,738 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=2 term=6 command=two 09:09:43,821 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:43,822 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 648.6 ms 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=3 term=6 command=three [WARN] [06/30/2025 09:09:43.822] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Append entry to log two 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=4 term=6 command=four 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Log size is now 2 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:09:43,822 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:09:43,822 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 648.3 ms [WARN] [06/30/2025 09:09:43.822] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [06/30/2025 09:09:43.822] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:09:43,822 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 2, nextIndex: 3 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 0 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 3 is present for follower follower787 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}]} 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=5 term=6 command=five 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 3, leaderLastIndex: 5, leaderSnapShotIndex: 0 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 3 is present for follower follower787 [WARN] [06/30/2025 09:09:43.823] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Persist data index=6 term=6 command=six 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 0 09:09:43,823 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 3 is present for follower follower787 [WARN] [06/30/2025 09:09:43.823] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 09:09:43,824 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:09:43,824 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 2.515 ms 09:09:43,824 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 4 commands after 4.100 μs 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 2 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 1 to 2 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Applying state for log index 2 data two 09:09:43,824 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader786: applyState called: two 09:09:43,825 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - leader786: fakeSnapshot purging log to 1 for term 6 09:09:43,826 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:09:43,826 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 3.569 ms 09:09:43,826 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@328c192 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}]} 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 2 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Append entry to log three 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Append entry to log four 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Log size is now 4 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=4, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 09:09:43,826 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 2 deferred callback(s) 09:09:43,826 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 2 commands after 348.2 μs 09:09:43,826 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 4, nextIndex: 5 09:09:43,826 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 1 09:09:43,827 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 5 is present for follower follower787 09:09:43,827 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=4, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:09:43,828 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 2 completion(s) pending 09:09:43,828 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: running 2 completion(s) 09:09:43,828 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 09:09:43,828 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 2 commands in 1.960 ms 09:09:43,828 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:43,828 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@2adaf370 09:09:43,828 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false} 09:09:43,828 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@6fbb93e8 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=4, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 2 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Append entry to log five 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Append entry to log six 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Log size is now 6 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Commit index set to 2 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 2 (of expected -1) 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 6 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 2 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 1 to 2 09:09:43,829 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 2 commands after 312.6 μs 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 2 data two 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower787: applyState called: two 09:09:43,829 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 6, nextIndex: 7 09:09:43,829 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower787: fakeSnapshot purging log to 1 for term 6 09:09:43,829 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 2 deferred callback(s) 09:09:43,829 AM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 4 completion(s) pending 09:09:43,829 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 4 commands in 5.167 ms 09:09:43,830 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 6.110 μs 09:09:43,830 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader786: running 4 completion(s) 09:09:43,830 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 09:09:43,830 AM [test-pekko.actor.default-dispatcher-12] [INFO] SnapshotManager - leader786: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=4, mandatoryTrim=false] 09:09:43,830 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 223.0 μs 09:09:43,830 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:43,830 AM [test-pekko.actor.default-dispatcher-11] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [ERROR] [06/30/2025 09:09:43.830] [test-pekko.actor.internal-dispatcher-4] [pekko://test/user/leader786] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:43,831 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#916,leader786-writer-174]/waiting 09:09:43,831 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 commands after 1.036 ms 09:09:43,831 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 11.43 μs 09:09:43,831 AM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: journal writer stopped 09:09:43,831 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: journal closed 09:09:43,831 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader786: journal closed 09:09:43,831 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:09:43,831 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:09:43,831 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 2 completion(s) pending 09:09:43,831 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: running 2 completion(s) 09:09:43,831 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 2 commands in 2.656 ms 09:09:43,831 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false} 09:09:43,831 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@2628309c 09:09:43,831 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 3.790 μs 09:09:43,832 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false} 09:09:43,832 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@153fc362 09:09:43,832 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 115.2 μs 09:09:43,832 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands [WARN] [06/30/2025 09:09:43.832] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@4dd45aeb 09:09:43,833 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 09:09:43,833 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: started executor 09:09:43,833 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: started executor 09:09:43,833 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=2 09:09:43,834 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=7 replayFrom=1 fileEntries=0 09:09:43,835 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=2 09:09:43,835 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: journal writer started 09:09:43,835 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRecovery - leader786: restoring local TermInfo{term=6, votedFor=leader786} 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@4c7400e 09:09:43,836 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:09:43,837 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:09:43,838 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader786: journal recovery completedin 1.411 ms with journalIndex=1 09:09:43,922 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:43,922 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:43,922 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:43,922 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,23 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,23 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,23 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,23 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,124 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,124 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,124 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,124 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,224 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,224 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,224 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,224 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,278 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:44,278 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 09:09:44,278 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 09:09:44,285 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 09:09:44,285 AM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader786 (Candidate): Starting new election term 7 09:09:44,285 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} to peer follower788 09:09:44,285 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} to peer follower787 09:09:44,285 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 [WARN] [06/30/2025 09:09:44.285] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} 09:09:44,285 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:44,285 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 09:09:44,285 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 09:09:44,285 AM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower787 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 09:09:44,288 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=7, votedFor=null 09:09:44,288 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 6, lastTerm: 6 09:09:44,290 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=7, votedFor=leader786 09:09:44,290 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 09:09:44,290 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 09:09:44,290 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 09:09:44,290 AM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader786 (Candidate): LastApplied index -1 is behind last index 6 - switching to PreLeader 09:09:44,290 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Election: Leader has following peers: [follower788, follower787] 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 [WARN] [06/30/2025 09:09:44.291] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=7, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:44,291 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Persist data index=7 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Replicate message: identifier: null, logIndex: 7, isSendImmediate: true 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:44,291 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: received 1 commands after 456.4 ms 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 6, nextIndex: 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 [WARN] [06/30/2025 09:09:44.291] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=7, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:44,291 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:44,291 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 7 is present for follower follower787 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Sending AppendEntries to follower follower787: AppendEntries{term=7, leaderId=leader786, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader786, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,292 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 7 is present for follower follower787 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:44,292 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 460.2 ms 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): Log size is now 6 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower787, logLastIndex=7, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:09:44,292 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 7, nextIndex: 8 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 09:09:44,292 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:44,293 AM [leader786-writer-175] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:09:44,293 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 2.180 ms 09:09:44,293 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:09:44,293 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:44,293 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:44,294 AM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader786: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=7, lastTerm=7, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=8, mandatoryTrim=false] 09:09:44,294 AM [test-pekko.actor.default-dispatcher-17] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:44,294 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#917,leader786-writer-175]/waiting [ERROR] [06/30/2025 09:09:44.294] [test-pekko.actor.internal-dispatcher-2] [pekko://test/user/leader786] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:349) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:44,294 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:09:44,295 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 2.545 ms 09:09:44,295 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:44,295 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: received 1 commands after 1.153 ms 09:09:44,295 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 8.770 μs 09:09:44,295 AM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: journal writer stopped 09:09:44,295 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:09:44,295 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader786: journal closed 09:09:44,295 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=false} 09:09:44,295 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader786: journal closed 09:09:44,295 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@443b5626 09:09:44,295 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:09:44,295 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks [WARN] [06/30/2025 09:09:44.295] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@2b0a857e 09:09:44,296 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 09:09:44,296 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: started executor 09:09:44,296 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: started executor 09:09:44,296 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=2 09:09:44,297 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=8 replayFrom=1 fileEntries=0 09:09:44,297 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=2 09:09:44,297 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: journal writer started 09:09:44,297 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRecovery - leader786: restoring local TermInfo{term=7, votedFor=leader786} 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3899f40d 09:09:44,299 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 09:09:44,300 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 09:09:44,300 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=1, term=6, command=one] 09:09:44,301 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=2, term=6, command=two] 09:09:44,301 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=3, term=6, command=three] 09:09:44,301 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=4, term=6, command=four] 09:09:44,301 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=5, term=6, command=five] 09:09:44,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=6, term=6, command=six] 09:09:44,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b] 09:09:44,302 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader786: journal recovery completedin 2.661 ms with journalIndex=1 09:09:44,325 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,325 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,325 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,325 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,425 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,426 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,526 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,627 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,727 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,727 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,727 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,727 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,788 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:44,788 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 09:09:44,788 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 09:09:44,792 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=8, votedFor=leader786 09:09:44,792 AM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader786 (Candidate): Starting new election term 8 09:09:44,792 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} to peer follower788 09:09:44,792 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} to peer follower787 09:09:44,792 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 8 [WARN] [06/30/2025 09:09:44.792] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} 09:09:44,792 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:44,792 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 09:09:44,792 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 09:09:44,793 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower787 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 09:09:44,795 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=8, votedFor=null 09:09:44,796 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 7, lastTerm: 7 09:09:44,798 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=8, votedFor=leader786 09:09:44,798 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=8, voteGranted=true}, my term is 8 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=8, voteGranted=true}, current voteCount: 1 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader786 (Candidate): LastApplied index -1 is behind last index 7 - switching to PreLeader 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Election: Leader has following peers: [follower788, follower787] 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 09:09:44,798 AM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=8 term=8 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,799 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:44,799 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 commands after 501.3 ms [WARN] [06/30/2025 09:09:44.799] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Replicate message: identifier: null, logIndex: 8, isSendImmediate: true 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 7, nextIndex: 8 [WARN] [06/30/2025 09:09:44.799] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 8 is present for follower follower787 09:09:44,799 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Sending AppendEntries to follower follower787: AppendEntries{term=8, leaderId=leader786, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=8, leaderId=leader786, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true}]} 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:44,799 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 09:09:44,799 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 8 is present for follower follower787 09:09:44,799 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 09:09:44,799 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,799 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:44,800 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Log size is now 7 09:09:44,800 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=8, success=true, followerId=follower787, logLastIndex=8, logLastTerm=8, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:44,800 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 09:09:44,800 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 09:09:44,800 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 505.1 ms 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 8, nextIndex: 9 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:44,800 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:44,801 AM [leader786-writer-176] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 09:09:44,801 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 1.961 ms 09:09:44,801 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=true} 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (PreLeader): Applying to log - commitIndex: 8, lastAppliedIndex: -1 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 0 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to 0 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 0 data zero 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: zero 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 1 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 0 to 1 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 1 data one 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: one 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 2 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 1 to 2 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 2 data two 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: two 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 3 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 2 to 3 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 3 data three 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: three 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 4 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 3 to 4 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 4 data four 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: four 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 5 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 4 to 5 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 5 data five 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: five 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 6 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 5 to 6 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 6 data six 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader786: applyState called: six 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 7 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 6 to 7 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 7 to 8 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader786: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,801 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 commands after 667.5 μs 09:09:44,801 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PreLeader - leader786 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 8, lastIndex: 8 09:09:44,801 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 78.83 μs 09:09:44,801 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting to receive commands 09:09:44,802 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (Leader): Election: Leader has following peers: [follower788, follower787] 09:09:44,802 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:44,802 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 09:09:44,802 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader786 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 8 [WARN] [06/30/2025 09:09:44.802] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:44,802 AM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b, pending=false} 09:09:44,802 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 2.453 ms 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@1b8443dd 09:09:44,802 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): Commit index set to 8 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 2, lastIndex(): 8 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 3 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 2 to 3 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 3 data three 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower787: applyState called: three 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 4 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 3 to 4 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 4 data four 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower787: applyState called: four 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 5 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 4 to 5 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 5 data five 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower787: applyState called: five 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 6 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 5 to 6 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 6 data six 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower787: applyState called: six 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 7 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 6 to 7 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 8 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 7 to 8 09:09:44,802 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower787: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2b41bf8b 09:09:44,802 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 248.7 μs 09:09:44,803 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 101.1 μs 09:09:44,803 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting to receive commands 09:09:44,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,828 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:44,919 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:44.919] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:44,928 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:44,928 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:44,928 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:44,928 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:45,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,29 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.039] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,39 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,129 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,129 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,129 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,129 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.158] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,158 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,230 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,230 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,230 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,230 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.278] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,279 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,330 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,331 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,331 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,331 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.398] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,398 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,431 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,431 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,431 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,431 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.519] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,519 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,532 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,532 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,532 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,532 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:45,632 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,632 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,632 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,632 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.638] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,638 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,733 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,733 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,733 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,733 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.759] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,759 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,833 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,833 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,833 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,833 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.878] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,879 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:45,934 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:45,934 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:45,934 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:45,934 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:45.999] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:45,999 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:46,34 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,34 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,34 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,34 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:46.118] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,118 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:46,135 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,135 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,135 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,135 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:46,236 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,236 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,236 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,236 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:46,239 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:46.239] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,336 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,336 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,336 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,336 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:46.358] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,358 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:46,437 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,437 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,437 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,437 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:46,479 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:46.479] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,538 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,538 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,538 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,538 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:46.599] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,599 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:46,638 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,638 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,638 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,638 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:46.718] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,718 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:46,739 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,739 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,739 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,739 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:46.839] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,839 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:46,839 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,839 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,839 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,839 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:46,940 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:46,940 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:46,940 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:46,940 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:46.959] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:46,959 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,40 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,40 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,40 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,40 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:47.078] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,78 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,141 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,141 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,141 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,141 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:47.199] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,199 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,241 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,241 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,241 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,242 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:47,318 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:47.318] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,342 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,342 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,342 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,342 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:47,438 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:47.438] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,443 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,443 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,443 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,443 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:47,543 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,543 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,543 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,543 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:47.559] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,559 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,644 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,644 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,644 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,644 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:47.678] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,678 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,744 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,744 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,744 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,744 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:47.798] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,798 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,845 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,845 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,845 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,845 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:47.919] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:47,919 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:47,945 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:47,945 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:47,945 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:47,945 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:48.038] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,39 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:48,46 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,46 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,46 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,46 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:48,146 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,146 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,146 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,146 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:48.158] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,158 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:48,247 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,247 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,247 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,247 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:48.278] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,279 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:48,347 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,347 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,347 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,348 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:48,398 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:48.398] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,448 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,448 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,448 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,448 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:48.518] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,518 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:48,549 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,549 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,549 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,549 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/30/2025 09:09:48.639] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,639 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 09:09:48,649 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,649 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,649 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,649 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:48,750 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/TermInfo.properties 09:09:48,750 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/journal-v1-1.log 09:09:48,750 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest15787035795357540957/leader786/metadata-v1-1.log 09:09:48,750 AM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 09:09:48,758 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/30/2025 09:09:48.758] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 09:09:48,851 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower787-collector789#-1030815767] 09:09:48,862 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower787] 09:09:48,862 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - follower787: terminating thread VirtualThread[#914,follower787-writer-172]/waiting 09:09:48,862 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 commands after 4.060 s 09:09:48,862 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: completed 1 commands in 16.33 μs 09:09:48,862 AM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: journal writer stopped 09:09:48,863 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower787: journal closed 09:09:48,863 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower787: journal closed 09:09:48,863 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower787: stopped executor with 0 remaining tasks 09:09:48,863 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower787: stopped executor with 0 remaining tasks 09:09:48,863 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower788-collector790#-829498932] 09:09:48,863 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower788] 09:09:48,863 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader786-collector791#-24822573] 09:09:48,863 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader786] 09:09:48,864 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#919,leader786-writer-176]/waiting 09:09:48,864 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 commands after 4.062 s 09:09:48,864 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: completed 1 commands in 11.87 μs 09:09:48,864 AM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: journal writer stopped 09:09:48,864 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader786: journal closed 09:09:48,864 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader786: journal closed 09:09:48,864 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:09:48,864 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 09:09:48,867 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotTriggeredByMemoryThresholdExceededWithLaggingFollower starting [WARN] [06/30/2025 09:09:48.868] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower796]: org.opendaylight.controller.cluster.common.actor.Monitor@7c4f2ce1 09:09:48,869 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - follower796: Starting recovery with journal batch size 1000 09:09:48,869 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower796: started executor 09:09:48,869 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower796: started executor 09:09:48,870 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower796: metadata journal open: replayFrom=1 applyTo=0 09:09:48,870 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower796: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:48,870 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower796: journal open: applyTo=0 09:09:48,870 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: journal writer started 09:09:48,870 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:09:48,871 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/follower796/TermInfo.properties does not exist 09:09:48,871 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/follower796/journal-v1-1.log 09:09:48,872 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/follower796/metadata-v1-1.log 09:09:48,872 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: no eligible files found 09:09:48,872 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower796: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:09:48.873] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower797]: org.opendaylight.controller.cluster.common.actor.Monitor@4d394881 09:09:48,873 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - follower797: Starting recovery with journal batch size 1000 09:09:48,873 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower797: started executor 09:09:48,873 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower797: started executor 09:09:48,873 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower797: metadata journal open: replayFrom=1 applyTo=0 09:09:48,874 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower797: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:48,874 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower797: journal open: applyTo=0 09:09:48,874 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: journal writer started 09:09:48,874 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting to receive commands 09:09:48,874 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/follower797/TermInfo.properties does not exist 09:09:48,874 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/follower797/journal-v1-1.log 09:09:48,874 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/follower797/metadata-v1-1.log 09:09:48,875 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: no eligible files found 09:09:48,875 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower797: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/30/2025 09:09:48.875] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader795]: org.opendaylight.controller.cluster.common.actor.Monitor@346f812d 09:09:48,875 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - leader795: Starting recovery with journal batch size 1000 09:09:48,875 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader795: started executor 09:09:48,875 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader795: started executor 09:09:48,875 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader795: metadata journal open: replayFrom=1 applyTo=0 09:09:48,876 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader795: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:09:48,876 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader795: journal open: applyTo=0 09:09:48,876 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: journal writer started 09:09:48,876 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:09:48,877 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=0, votedFor=null 09:09:48,877 AM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower796: Local TermInfo store seeded with TermInfo{term=0} 09:09:48,877 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower796: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@696eaa4e 09:09:48,877 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from -1 to -1 09:09:48,877 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/leader795/TermInfo.properties does not exist 09:09:48,877 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader795: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/leader795/journal-v1-1.log 09:09:48,877 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader795: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3662991315455827001/leader795/metadata-v1-1.log 09:09:48,877 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader795: no eligible files found 09:09:48,877 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=5, votedFor=leader795 09:09:48,877 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader795: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:09:48,877 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower796: journal recovery completedin 260.0 μs with journalIndex=1 09:09:48,880 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=0, votedFor=null 09:09:48,880 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower797: Local TermInfo store seeded with TermInfo{term=0} 09:09:48,880 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower797: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@299e9d05 09:09:48,880 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower797: Moving last applied index from -1 to -1 09:09:48,880 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - follower797: journal recovery completedin 122.9 μs with journalIndex=1 09:09:48,881 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=5, votedFor=leader795 09:09:48,881 AM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader795: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader795} 09:09:48,881 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader795: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@504850df 09:09:48,881 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from -1 to -1 09:09:48,881 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader795: journal recovery completedin 158.2 μs with journalIndex=1 09:09:49,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:09:49,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader795 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.168:17355, Removed) 09:09:49,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader795 (Follower): Received ElectionTimeout - switching to Candidate 09:09:49,388 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader795 (Candidate): Election: Candidate has following voting peers: [follower797, follower796] 09:09:49,392 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=6, votedFor=leader795 09:09:49,392 AM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader795 (Candidate): Starting new election term 6 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader795 (Candidate): Sending RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} to peer follower797 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader795 (Candidate): Sending RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} to peer follower796 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader795 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower797 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower797 (Follower): Cluster state: [] 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower797 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:49,393 AM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower797 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:49,393 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:09:49,393 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): Cluster state: [] 09:09:49,393 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 09:09:49,393 AM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower796 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:09:49,396 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=6, votedFor=null 09:09:49,396 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower797 (Follower): In requestVote: RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:49,398 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=6, votedFor=null 09:09:49,398 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): In requestVote: RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:09:49,398 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=6, votedFor=leader795 09:09:49,398 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower797 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:49,398 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader795 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 09:09:49,398 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader795 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): Election: Leader has following peers: [follower797, follower796] 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader795 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower797: Last sync leader does not match current leader leader795, need to catch up to -1 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower797 updated: matchIndex: -1, nextIndex: 0 09:09:49,399 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:49.399] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower797] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower797, initialSyncDone=false] 09:09:49,401 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=6, votedFor=leader795 09:09:49,401 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 09:09:49,401 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Last sync leader does not match current leader leader795, need to catch up to -1 09:09:49,401 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: -1, nextIndex: 0 09:09:49,401 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/30/2025 09:09:49.401] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower796] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower796, initialSyncDone=false] 09:09:49,434 AM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 09:09:49,434 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader795: Persist data index=0 term=6 command=zero 09:09:49,434 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader795: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:49,434 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 09:09:49,434 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:49,434 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 0 is present for follower follower797 09:09:49,434 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 558.4 ms 09:09:49,434 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower797: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): Number of entries to be appended = 1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): Append entry to log zero 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower797: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): Log size is now 1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower797 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower797, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached -1 (of expected -1) 09:09:49,435 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 commands after 560.8 ms 09:09:49,435 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower797: awaiting more completions to resolve 1 deferred callback(s) 09:09:49,435 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 0 is present for follower follower796 09:09:49,435 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower796: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower797 updated: matchIndex: 0, nextIndex: 1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower796 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower796 (Follower): Number of entries to be appended = 1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower796 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower796 (Follower): Append entry to log zero 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower796: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower796 (Follower): Log size is now 1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower796 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower796, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached -1 (of expected -1) 09:09:49,435 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 564.8 ms 09:09:49,435 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower796: awaiting more completions to resolve 1 deferred callback(s) 09:09:49,435 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: 0, nextIndex: 1 09:09:49,435 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:09:49,440 AM [leader795-writer-179] [DEBUG] EntryStoreCompleter - leader795: 1 completion(s) pending 09:09:49,440 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 5.146 ms 09:09:49,440 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader795: running 1 completion(s) 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader795: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader795 (Leader): Setting last applied to 0 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from -1 to 0 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader795: Applying state for log index 0 data zero 09:09:49,440 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader795: applyState called: zero 09:09:49,440 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 323.7 μs 09:09:49,440 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 174.5 μs 09:09:49,440 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:09:49,441 AM [follower796-writer-177] [DEBUG] EntryStoreCompleter - follower796: 1 completion(s) pending 09:09:49,441 AM [follower797-writer-178] [DEBUG] EntryStoreCompleter - follower797: 1 completion(s) pending 09:09:49,441 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 5.824 ms 09:09:49,441 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:09:49,441 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: completed 1 commands in 6.208 ms 09:09:49,441 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting to receive commands 09:09:49,441 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower796: running 1 completion(s) 09:09:49,441 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower797: running 1 completion(s) 09:09:49,441 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower796: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:49,441 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower797: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 09:09:49,441 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower796: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@11afef9d 09:09:49,441 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower797: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@2a325024 [WARN] [06/30/2025 09:09:49.441] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower796] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower796, initialSyncDone=true] [WARN] [06/30/2025 09:09:49.441] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower797] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower797, initialSyncDone=true] 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Commit index set to 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): Commit index set to 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower797 (Follower): Setting last applied to 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower797: Moving last applied index from -1 to 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower797: Applying state for log index 0 data zero 09:09:49,639 AM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower797: applyState called: zero 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): Setting last applied to 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from -1 to 0 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower796: Applying state for log index 0 data zero 09:09:49,639 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower796: applyState called: zero 09:09:49,639 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 commands after 198.2 ms 09:09:49,639 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 198.2 ms 09:09:49,639 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 129.7 μs 09:09:49,639 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: completed 1 commands in 139.4 μs 09:09:49,639 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:09:49,639 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting to receive commands 09:09:49,758 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:49,758 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:49,878 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:49,999 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:50,118 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader795: Persist data index=1 term=6 command=one 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader795: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:50,193 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 753.0 ms 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 1 is present for follower follower796 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower796: AppendEntries{term=6, leaderId=leader795, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:50,193 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Number of entries to be appended = 1 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Append entry to log one 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower796: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:50,194 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 554.3 ms 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Log size is now 2 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower796, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 09:09:50,194 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower796: awaiting more completions to resolve 1 deferred callback(s) 09:09:50,194 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: 1, nextIndex: 2 09:09:50,194 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:09:50,197 AM [leader795-writer-179] [DEBUG] EntryStoreCompleter - leader795: 1 completion(s) pending 09:09:50,197 AM [follower796-writer-177] [DEBUG] EntryStoreCompleter - follower796: 1 completion(s) pending 09:09:50,197 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 4.037 ms 09:09:50,197 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 3.568 ms 09:09:50,197 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:09:50,197 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:09:50,197 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower796: running 1 completion(s) 09:09:50,197 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - leader795: running 1 completion(s) 09:09:50,197 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower796: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 09:09:50,197 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader795: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 09:09:50,198 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower796: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087ad08@1d13b973 09:09:50,198 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader795 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:09:50,198 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader795 (Leader): Setting last applied to 1 09:09:50,198 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from 0 to 1 09:09:50,198 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader795: Applying state for log index 1 data one 09:09:50,198 AM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader795: applyState called: one 09:09:50,198 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader795: fakeSnapshot purging log to 0 for term 6 09:09:50,198 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 591.8 μs 09:09:50,198 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 123.1 μs 09:09:50,198 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting to receive commands 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Commit index set to 1 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower796 (Follower): Setting last applied to 1 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from 0 to 1 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower796: Applying state for log index 1 data one 09:09:50,358 AM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower796: applyState called: one 09:09:50,359 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower796: fakeSnapshot purging log to 0 for term 6 09:09:50,359 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 161.2 ms 09:09:50,359 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 113.8 μs 09:09:50,359 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting to receive commands 09:09:50,479 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:50,598 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:50,718 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:50,838 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:50,959 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,78 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,198 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,319 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,438 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,559 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,678 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,799 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:51,909 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,19 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,129 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,248 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,358 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,469 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,588 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,708 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,829 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:52,948 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,68 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,189 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,308 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,429 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,539 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,658 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,779 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:53,898 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,8 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,129 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,248 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,359 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,478 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,598 AM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,709 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,828 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:54,948 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:55,58 AM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:55,179 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 09:09:55,244 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower796-collector798#-512549769] 09:09:55,245 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower796] 09:09:55,245 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower796: terminating thread VirtualThread[#920,follower796-writer-177]/waiting 09:09:55,245 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 commands after 4.887 s 09:09:55,245 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: completed 1 commands in 14.82 μs 09:09:55,245 AM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: journal writer stopped 09:09:55,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower796: journal closed 09:09:55,246 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower796: journal closed 09:09:55,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower796: stopped executor with 0 remaining tasks 09:09:55,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower796: stopped executor with 0 remaining tasks 09:09:55,246 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower797-collector799#-1562496530] 09:09:55,246 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower797] 09:09:55,246 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower797: terminating thread VirtualThread[#921,follower797-writer-178]/waiting 09:09:55,246 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 commands after 5.607 s 09:09:55,246 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: completed 1 commands in 11.51 μs 09:09:55,246 AM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: journal writer stopped 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower797: journal closed 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower797: journal closed 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: stopped executor with 0 remaining tasks 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower797: stopped executor with 0 remaining tasks 09:09:55,247 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader795-collector800#6459985] 09:09:55,247 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader795] 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader795: terminating thread VirtualThread[#922,leader795-writer-179]/waiting 09:09:55,247 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 commands after 5.049 s 09:09:55,247 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: completed 1 commands in 11.34 μs 09:09:55,247 AM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: journal writer stopped 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader795: journal closed 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader795: journal closed 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader795: stopped executor with 0 remaining tasks 09:09:55,247 AM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader795: stopped executor with 0 remaining tasks [INFO] [06/30/2025 09:09:55.252] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 09:09:55,252 AM [test-pekko.actor.default-dispatcher-15] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [WARN] [06/30/2025 09:09:55.253] [test-pekko.actor.default-dispatcher-15] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.170.168:17355) [INFO] [06/30/2025 09:09:55.254] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - Exiting completed 09:09:55,254 AM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - Exiting completed [INFO] [06/30/2025 09:09:55.254] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - Shutting down... 09:09:55,254 AM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - Shutting down... [INFO] [06/30/2025 09:09:55.254] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.168:17355] - Successfully shut down 09:09:55,254 AM [test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.168:17355] - Successfully shut down [INFO] [06/30/2025 09:09:55.255] [test-pekko.remote.default-remote-dispatcher-7] [pekko://test@10.30.170.168:17355/system/remoting-terminator] Shutting down remote daemon. 09:09:55,255 AM [test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. [INFO] [06/30/2025 09:09:55.255] [test-pekko.remote.default-remote-dispatcher-7] [pekko://test@10.30.170.168:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports. 09:09:55,255 AM [test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:55,260 AM [test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [INFO] [06/30/2025 09:09:55.260] [test-pekko.remote.default-remote-dispatcher-9] [pekko://test@10.30.170.168:17355/system/remoting-terminator] Remoting shut down. [DEBUG] [06/30/2025 09:09:55.260] [test-pekko.actor.internal-dispatcher-3] [EventStream] shutting down: StandardOutLogger