[DEBUG] [07/01/2025 13:07:14.263] [main] [EventStream(pekko://test)] logger log1-TestEventListener started
01:07:14,264 PM [test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started
[DEBUG] [07/01/2025 13:07:14.264] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started
[DEBUG] [07/01/2025 13:07:14.264] [main] [EventStream(pekko://test)] Default Loggers started
01:07:14,281 PM [test-pekko.actor.default-dispatcher-7] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.171.139:17355] with UID [4445205913398835966]
[INFO] [07/01/2025 13:07:14.281] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.171.139:17355] with UID [4445205913398835966]
[INFO] [07/01/2025 13:07:14.281] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - Starting up, Pekko version [1.0.3] ...
01:07:14,282 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - Starting up, Pekko version [1.0.3] ...
01:07:14,284 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - Registered cluster JMX MBean [pekko:type=Cluster]
01:07:14,284 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - Started up successfully
01:07:14,285 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing
01:07:14,285 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining
[INFO] [07/01/2025 13:07:14.284] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - Registered cluster JMX MBean [pekko:type=Cluster]
[INFO] [07/01/2025 13:07:14.284] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - Started up successfully
[INFO] [07/01/2025 13:07:14.285] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing
[INFO] [07/01/2025 13:07:14.285] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining
01:07:14,294 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries starting: sending 2 new payloads
[WARN] [07/01/2025 13:07:14.296] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower761]: org.opendaylight.controller.cluster.common.actor.Monitor@61730f34
[INFO] [pekkoDeadLetter][07/01/2025 13:07:14.296] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,297 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,298 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - follower761: Starting recovery with journal batch size 1000
01:07:14,299 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: started executor
01:07:14,299 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: started executor
01:07:14,299 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower761: metadata journal open: replayFrom=1 applyTo=0
01:07:14,300 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower761: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:14,300 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower761: journal open: applyTo=0
01:07:14,300 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: journal writer started
01:07:14,300 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: waiting to receive commands
[WARN] [07/01/2025 13:07:14.301] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower762]: org.opendaylight.controller.cluster.common.actor.Monitor@3484a697
01:07:14,301 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/follower761/TermInfo.properties does not exist
01:07:14,301 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/follower761/journal-v1-1.log
01:07:14,301 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/follower761/metadata-v1-1.log
01:07:14,301 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower761: no eligible files found
01:07:14,301 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - follower761: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:14,302 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - follower762: Starting recovery with journal batch size 1000
01:07:14,302 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower762: started executor
01:07:14,302 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower762: started executor
[INFO] [pekkoDeadLetter][07/01/2025 13:07:14.302] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower762] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,302 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower762] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,305 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower762: metadata journal open: replayFrom=1 applyTo=0
01:07:14,305 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower762: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:14,305 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower762: journal open: applyTo=0
01:07:14,305 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: journal writer started
01:07:14,305 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: waiting to receive commands
[WARN] [07/01/2025 13:07:14.306] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader760]: org.opendaylight.controller.cluster.common.actor.Monitor@6bb5bfe1
[INFO] [pekkoDeadLetter][07/01/2025 13:07:14.306] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader760] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,307 PM [test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader760] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/follower762/TermInfo.properties does not exist
01:07:14,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower762: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/follower762/journal-v1-1.log
01:07:14,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower762: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/follower762/metadata-v1-1.log
01:07:14,307 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - leader760: Starting recovery with journal batch size 1000
01:07:14,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower762: no eligible files found
01:07:14,307 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower762: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:14,307 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader760: started executor
01:07:14,307 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader760: started executor
01:07:14,307 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader760: metadata journal open: replayFrom=1 applyTo=0
01:07:14,307 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader760: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:14,307 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader760: journal open: applyTo=0
01:07:14,308 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: journal writer started
01:07:14,308 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: waiting to receive commands
01:07:14,309 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/leader760/TermInfo.properties does not exist
01:07:14,309 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/leader760/journal-v1-1.log
01:07:14,309 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest4148808363870276670/leader760/metadata-v1-1.log
01:07:14,309 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader760: no eligible files found
01:07:14,309 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader760: Set currentTerm=5, votedFor=leader760
01:07:14,309 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader760: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:14,320 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=0, votedFor=null
01:07:14,320 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - follower761: Local TermInfo store seeded with TermInfo{term=0}
01:07:14,320 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower761: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@485965e5
01:07:14,320 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to -1
01:07:14,321 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - follower761: journal recovery completedin 205.0 μs with journalIndex=1
01:07:14,325 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader760: Set currentTerm=5, votedFor=leader760
01:07:14,325 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader760: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader760}
01:07:14,325 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader760: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@317d7a3e
01:07:14,325 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader760: Moving last applied index from -1 to -1
01:07:14,325 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader760: journal recovery completedin 293.3 μs with journalIndex=1
01:07:14,325 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower762: Set currentTerm=0, votedFor=null
01:07:14,325 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower762: Local TermInfo store seeded with TermInfo{term=0}
01:07:14,325 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower762: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@32c4c7fd
01:07:14,325 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower762: Moving last applied index from -1 to -1
01:07:14,325 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - follower762: journal recovery completedin 161.0 μs with journalIndex=1
01:07:14,809 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
01:07:14,809 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Follower - leader760 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.139:17355, Removed)
01:07:14,810 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Follower - leader760 (Follower): Received ElectionTimeout - switching to Candidate
01:07:14,810 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Candidate - leader760 (Candidate): Election: Candidate has following voting peers: [follower762, follower761]
01:07:14,858 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] PropertiesTermInfoStore - leader760: Set currentTerm=6, votedFor=leader760
01:07:14,858 PM [test-pekko.actor.default-dispatcher-13] [INFO] Candidate - leader760 (Candidate): Starting new election term 6
01:07:14,858 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Candidate - leader760 (Candidate): Sending RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1} to peer follower762
01:07:14,858 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Candidate - leader760 (Candidate): Sending RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1} to peer follower761
01:07:14,858 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower762 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:14,858 PM [test-pekko.actor.default-dispatcher-13] [INFO] RaftActorBehavior - leader760 (Follower) :- Switching from behavior Follower to Candidate, election term: 6
01:07:14,858 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower762 (Follower): Cluster state: []
01:07:14,858 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower762 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:14,858 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower762 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:14,859 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower761 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:14,859 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower761 (Follower): Cluster state: []
01:07:14,859 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower761 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:14,859 PM [test-pekko.actor.default-dispatcher-13] [INFO] Follower - follower761 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:14,903 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=null
01:07:14,903 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower761 (Follower): In requestVote: RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:14,931 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower762: Set currentTerm=6, votedFor=null
01:07:14,931 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower762 (Follower): In requestVote: RequestVote{term=6, candidateId=leader760, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=leader760
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower761 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Candidate - leader760 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Candidate - leader760 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractLeader - leader760 (Leader): Election: Leader has following peers: [follower762, follower761]
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower762: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower761: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:14,965 PM [test-pekko.actor.default-dispatcher-13] [INFO] RaftActorBehavior - leader760 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6
01:07:14,966 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower761: Last sync leader does not match current leader leader760, need to catch up to -1
01:07:14,966 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractLeader - leader760 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: -1, nextIndex: 0
01:07:14,966 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [07/01/2025 13:07:14.966] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=false]
[INFO] [pekkoDeadLetter][07/01/2025 13:07:14.966] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,966 PM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:14,997 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower762: Set currentTerm=6, votedFor=leader760
01:07:14,997 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower762 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:14,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower762: Last sync leader does not match current leader leader760, need to catch up to -1
01:07:14,998 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower762 updated: matchIndex: -1, nextIndex: 0
01:07:14,998 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower762: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:14,998 PM [test-pekko.actor.default-dispatcher-6] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower762] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [07/01/2025 13:07:14.998] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower762] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower762, initialSyncDone=false]
[INFO] [pekkoDeadLetter][07/01/2025 13:07:14.998] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower762] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower762] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,10 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected
01:07:15,10 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader760: Persist data index=0 term=6 command=zero
01:07:15,10 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader760: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:15,10 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: received 1 commands after 702.5 ms
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower762: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 0 is present for follower follower762
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Sending AppendEntries to follower follower762: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 0 is present for follower follower761
01:07:15,11 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): Number of entries to be appended = 1
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): Append entry to log zero
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): Log size is now 1
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower761, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached -1 (of expected -1)
01:07:15,11 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: received 1 commands after 711.6 ms
[WARN] [07/01/2025 13:07:15.011] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=true]
[INFO] [pekkoDeadLetter][07/01/2025 13:07:15.011] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,12 PM [test-pekko.actor.default-dispatcher-6] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,12 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader760: awaiting more completions to resolve 1 synchronized callback(s)
01:07:15,33 PM [leader760-writer-161] [DEBUG] RaftStorageCompleter - leader760: 1 completion(s) pending
01:07:15,33 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: completed 1 commands in 22.24 ms
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader760: running 1 completion(s)
01:07:15,33 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: waiting to receive commands
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader760: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true}
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader760: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@6350dcb5
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: 0, nextIndex: 1
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader760 (Leader): Setting last applied to 0
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader760: Moving last applied index from -1 to 0
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader760: Applying state for log index 0 data zero
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader760: applyState called: zero
01:07:15,33 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: received 1 commands after 816.1 μs
01:07:15,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,34 PM [leader760-writer-161] [DEBUG] RaftStorageCompleter - leader760: 1 completion(s) pending
01:07:15,34 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: completed 1 commands in 153.3 μs
01:07:15,34 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: waiting to receive commands
01:07:15,34 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader760: running 1 completion(s)
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Follower - follower761 (Follower): Commit index set to 0
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached 0 (of expected -1)
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader760: Persist data index=1 term=6 command=one
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Follower - follower761 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader760: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 0
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to 0
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActor - follower761: Applying state for log index 0 data zero
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
01:07:15,34 PM [test-pekko.actor.default-dispatcher-13] [INFO] MockRaftActor - follower761: applyState called: zero
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower762: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 0 is present for follower follower762
01:07:15,34 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: received 1 commands after 269.5 μs
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Sending AppendEntries to follower follower762: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 1 is present for follower follower761
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader760, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,34 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader760: awaiting more completions to resolve 1 synchronized callback(s)
01:07:15,34 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader760, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,34 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): Number of entries to be appended = 1
01:07:15,34 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:15,34 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): Append entry to log one
01:07:15,34 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:15,34 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): Log size is now 2
01:07:15,35 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower761 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower761, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,35 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached 0 (of expected -1)
01:07:15,43 PM [follower761-writer-159] [DEBUG] RaftStorageCompleter - follower761: 1 completion(s) pending
01:07:15,43 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 31.99 ms
01:07:15,43 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower761: running 1 completion(s)
01:07:15,43 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: received 2 commands after 7.071 μs
01:07:15,43 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:15,50 PM [leader760-writer-161] [DEBUG] RaftStorageCompleter - leader760: 1 completion(s) pending
01:07:15,50 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: completed 1 commands in 15.73 ms
01:07:15,50 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: waiting to receive commands
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader760: running 1 completion(s)
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader760: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true}
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader760: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@61617a24
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 1 is present for follower follower761
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: 1, nextIndex: 2
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader760 (Leader): Setting last applied to 1
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader760: Moving last applied index from 0 to 1
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader760: Applying state for log index 1 data one
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader760: applyState called: one
01:07:15,50 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower761 (Follower): Commit index set to 1
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached 1 (of expected -1)
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower761 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 1
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from 0 to 1
01:07:15,51 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: received 1 commands after 816.7 μs
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower761: Applying state for log index 1 data one
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower761: applyState called: one
01:07:15,51 PM [leader760-writer-161] [DEBUG] RaftStorageCompleter - leader760: 1 completion(s) pending
01:07:15,51 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: completed 1 commands in 217.3 μs
01:07:15,51 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: waiting to receive commands
01:07:15,51 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader760: running 1 completion(s)
01:07:15,61 PM [follower761-writer-159] [DEBUG] RaftStorageCompleter - follower761: 2 completion(s) pending
01:07:15,61 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: completed 2 commands in 17.35 ms
01:07:15,61 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower761: running 2 completion(s)
01:07:15,61 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: received 1 commands after 6.430 μs
01:07:15,61 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:15,61 PM [follower761-writer-159] [DEBUG] RaftStorageCompleter - follower761: 1 completion(s) pending
01:07:15,61 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 148.3 μs
01:07:15,61 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower761: running 1 completion(s)
01:07:15,61 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: waiting to receive commands
01:07:15,199 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 0 is present for follower follower762
01:07:15,199 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader760 (Leader): Sending AppendEntries to follower follower762: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]}
01:07:15,199 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached 1 (of expected -1)
01:07:15,212 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries: new entries applied - resuming follower follower762
01:07:15,318 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader760 (Leader): sendAppendEntries: 0 is present for follower follower762
01:07:15,319 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader760 (Leader): Sending AppendEntries to follower follower762: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]}
01:07:15,319 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached 1 (of expected -1)
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader760, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]}
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): Number of entries to be appended = 2
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): Append entry to log zero
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower762: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=false} 
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): Append entry to log one
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower762: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=false} 
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): Log size is now 2
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): Commit index set to 1
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower762, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,319 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: received 2 commands after 1.014 s
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower762: Lagging 0 entries behind leader leader760 and reached 1 (of expected -1)
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower762 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower762 (Follower): Setting last applied to 0
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower762: Moving last applied index from -1 to 0
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower762: Applying state for log index 0 data zero
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower762: applyState called: zero
01:07:15,319 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower762 (Follower): Setting last applied to 1
01:07:15,320 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower762: Moving last applied index from 0 to 1
01:07:15,320 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower762: Applying state for log index 1 data one
01:07:15,320 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower762: applyState called: one
01:07:15,320 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader760 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower762 updated: matchIndex: 1, nextIndex: 2
01:07:15,320 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader760: fakeSnapshot purging log to 0 for term 6
01:07:15,320 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader760 (Leader): Checking sendAppendEntries for follower follower762: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:15,320 PM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower762] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [07/01/2025 13:07:15.320] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower762] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower762, initialSyncDone=true]
[INFO] [pekkoDeadLetter][07/01/2025 13:07:15.320] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower762] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower762] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,349 PM [follower762-writer-160] [DEBUG] RaftStorageCompleter - follower762: 2 completion(s) pending
01:07:15,349 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: completed 2 commands in 30.18 ms
01:07:15,350 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: received 1 commands after 8.401 μs
01:07:15,350 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower762: running 2 completion(s)
01:07:15,350 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower762: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:15,350 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower762: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:15,350 PM [follower762-writer-160] [DEBUG] RaftStorageCompleter - follower762: 1 completion(s) pending
01:07:15,350 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: completed 1 commands in 221.9 μs
01:07:15,350 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: waiting to receive commands
01:07:15,350 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower762: running 1 completion(s)
01:07:15,439 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader760 and reached 1 (of expected -1)
01:07:15,439 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower762: Lagging 0 entries behind leader leader760 and reached 1 (of expected -1)
01:07:15,439 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower761: fakeSnapshot purging log to 0 for term 6
01:07:15,439 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - follower762: fakeSnapshot purging log to 0 for term 6
01:07:15,464 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries complete
01:07:15,464 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower761-collector763#1370171166]
01:07:15,475 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower761]
01:07:15,475 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower761: terminating thread VirtualThread[#926,follower761-writer-159]/waiting
01:07:15,475 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: received 1 commands after 414.3 ms
01:07:15,475 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: completed 1 commands in 27.64 μs
01:07:15,475 PM [follower761-writer-159] [DEBUG] JournalWriteTask - follower761: journal writer stopped
01:07:15,476 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower761: journal closed
01:07:15,476 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower761: journal closed
01:07:15,476 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks
01:07:15,476 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks
01:07:15,476 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower762-collector764#-780992149]
01:07:15,476 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower762]
01:07:15,476 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower762: terminating thread VirtualThread[#928,follower762-writer-160]/waiting
01:07:15,476 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: received 1 commands after 126.7 ms
01:07:15,477 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: completed 1 commands in 17.88 μs
01:07:15,477 PM [follower762-writer-160] [DEBUG] JournalWriteTask - follower762: journal writer stopped
01:07:15,477 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower762: journal closed
01:07:15,477 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower762: journal closed
01:07:15,477 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower762: stopped executor with 0 remaining tasks
01:07:15,477 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower762: stopped executor with 0 remaining tasks
01:07:15,477 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader760-collector765#1572854855]
01:07:15,477 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader760]
01:07:15,477 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - leader760: terminating thread VirtualThread[#930,leader760-writer-161]/waiting
01:07:15,477 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: received 1 commands after 426.6 ms
01:07:15,477 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: completed 1 commands in 18.89 μs
01:07:15,477 PM [leader760-writer-161] [DEBUG] JournalWriteTask - leader760: journal writer stopped
01:07:15,478 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader760: journal closed
01:07:15,478 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader760: journal closed
01:07:15,478 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader760: stopped executor with 0 remaining tasks
01:07:15,478 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader760: stopped executor with 0 remaining tasks
01:07:15,480 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaInstallSnapshot starting
[WARN] [07/01/2025 13:07:15.481] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower770]: org.opendaylight.controller.cluster.common.actor.Monitor@354b76aa
[INFO] [pekkoDeadLetter][07/01/2025 13:07:15.481] [test-pekko.actor.default-dispatcher-13] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,481 PM [test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,482 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower770: Starting recovery with journal batch size 1000
01:07:15,482 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower770: started executor
01:07:15,482 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower770: started executor
01:07:15,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower770: metadata journal open: replayFrom=1 applyTo=0
01:07:15,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower770: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:15,483 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower770: journal open: applyTo=0
01:07:15,483 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: journal writer started
01:07:15,484 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:15,484 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower770/TermInfo.properties does not exist
01:07:15,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower770/journal-v1-1.log
01:07:15,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower770/metadata-v1-1.log
01:07:15,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower770: no eligible files found
01:07:15,485 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower770: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:15,485 PM [test-pekko.actor.default-dispatcher-13] [INFO] RaftActor - follower771: Starting recovery with journal batch size 1000
01:07:15,485 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorage - follower771: started executor
01:07:15,485 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorage - follower771: started executor
[WARN] [07/01/2025 13:07:15.485] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower771]: org.opendaylight.controller.cluster.common.actor.Monitor@241135ad
[INFO] [pekkoDeadLetter][07/01/2025 13:07:15.486] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower771] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,486 PM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower771] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,486 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] EntryJournalV1 - follower771: metadata journal open: replayFrom=1 applyTo=0
01:07:15,486 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] EntryJournalV1 - follower771: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:15,486 PM [test-pekko.actor.default-dispatcher-13] [INFO] PekkoRaftStorage - follower771: journal open: applyTo=0
01:07:15,487 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: journal writer started
01:07:15,487 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: waiting to receive commands
01:07:15,488 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/TermInfo.properties does not exist
01:07:15,489 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - leader769: Starting recovery with journal batch size 1000
[WARN] [07/01/2025 13:07:15.489] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader769]: org.opendaylight.controller.cluster.common.actor.Monitor@602c7480
01:07:15,489 PM [test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader769] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[INFO] [pekkoDeadLetter][07/01/2025 13:07:15.489] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader769] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
01:07:15,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader769: started executor
01:07:15,490 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorage - follower771: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/journal-v1-1.log
01:07:15,490 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorage - follower771: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/metadata-v1-1.log
01:07:15,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader769: started executor
01:07:15,490 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorage - follower771: no eligible files found
01:07:15,490 PM [test-pekko.actor.default-dispatcher-13] [INFO] PekkoRecovery - follower771: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:15,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader769: metadata journal open: replayFrom=1 applyTo=0
01:07:15,491 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader769: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:15,491 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader769: journal open: applyTo=0
01:07:15,491 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: journal writer started
01:07:15,491 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:15,492 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties does not exist
01:07:15,492 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:15,492 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:15,492 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader769: no eligible files found
01:07:15,492 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader769: Set currentTerm=5, votedFor=leader769
01:07:15,492 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader769: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:15,498 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=0, votedFor=null
01:07:15,498 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower770: Local TermInfo store seeded with TermInfo{term=0}
01:07:15,498 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower770: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1ce37cf1
01:07:15,498 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to -1
01:07:15,498 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower770: journal recovery completedin 235.2 μs with journalIndex=1
01:07:15,536 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] PropertiesTermInfoStore - follower771: Set currentTerm=0, votedFor=null
01:07:15,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader769: Set currentTerm=5, votedFor=leader769
01:07:15,536 PM [test-pekko.actor.default-dispatcher-13] [INFO] PekkoRecovery - follower771: Local TermInfo store seeded with TermInfo{term=0}
01:07:15,536 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader769: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader769}
01:07:15,536 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActor - follower771: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7734abaf
01:07:15,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader769: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@31582778
01:07:15,536 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractReplicatedLog - follower771: Moving last applied index from -1 to -1
01:07:15,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from -1 to -1
01:07:15,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - leader769: journal recovery completedin 258.1 μs with journalIndex=1
01:07:15,536 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] JournalRecovery - follower771: journal recovery completedin 321.8 μs with journalIndex=1
01:07:15,979 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
01:07:15,979 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - leader769 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.139:17355, Removed)
01:07:15,979 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - leader769 (Follower): Received ElectionTimeout - switching to Candidate
01:07:15,979 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader769 (Candidate): Election: Candidate has following voting peers: [follower771, follower770]
01:07:15,982 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader769: Set currentTerm=6, votedFor=leader769
01:07:15,982 PM [test-pekko.actor.default-dispatcher-7] [INFO] Candidate - leader769 (Candidate): Starting new election term 6
01:07:15,982 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader769 (Candidate): Sending RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1} to peer follower771
01:07:15,982 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader769 (Candidate): Sending RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1} to peer follower770
01:07:15,982 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower771 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:15,982 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:15,982 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader769 (Follower) :- Switching from behavior Follower to Candidate, election term: 6
01:07:15,982 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower771 (Follower): Cluster state: []
01:07:15,982 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: []
01:07:15,982 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower771 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:15,982 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:15,982 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower771 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:15,982 PM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower770 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:15,985 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=null
01:07:15,985 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:15,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower771: Set currentTerm=6, votedFor=null
01:07:15,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower771 (Follower): In requestVote: RequestVote{term=6, candidateId=leader769, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=leader769
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader769 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader769 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader769 (Leader): Election: Leader has following peers: [follower771, follower770]
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:15,987 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader769 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6
01:07:15,987 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] SyncStatusTracker - follower770: Last sync leader does not match current leader leader769, need to catch up to -1
01:07:15,988 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: -1, nextIndex: 0
01:07:15,988 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [07/01/2025 13:07:15.988] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=false]
01:07:15,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower771: Set currentTerm=6, votedFor=leader769
01:07:15,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower771 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:15,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower771: Last sync leader does not match current leader leader769, need to catch up to -1
01:07:15,988 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower771 updated: matchIndex: -1, nextIndex: 0
01:07:15,989 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [07/01/2025 13:07:15.989] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower771] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower771, initialSyncDone=false]
01:07:15,990 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader769: Persist data index=0 term=6 command=zero
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 0 is present for follower follower771
01:07:15,990 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 499.4 ms
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower771: AppendEntries{term=6, leaderId=leader769, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 0 is present for follower follower770
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,990 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:15,990 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,990 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Append entry to log zero
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Log size is now 1
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,991 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached -1 (of expected -1)
01:07:15,991 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 507.1 ms
[WARN] [07/01/2025 13:07:15.991] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=true]
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): Number of entries to be appended = 1
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): Append entry to log zero
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower771: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): Log size is now 1
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower771, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,991 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached -1 (of expected -1)
01:07:15,991 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: received 1 commands after 504.2 ms
[WARN] [07/01/2025 13:07:15.991] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower771] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower771, initialSyncDone=true]
01:07:15,996 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:15,996 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 5.588 ms
01:07:15,996 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true}
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@65732b1f
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 0, nextIndex: 1
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 0
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from -1 to 0
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader769: Applying state for log index 0 data zero
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader769: applyState called: zero
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower771 updated: matchIndex: 0, nextIndex: 1
01:07:15,996 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 488.6 μs
01:07:15,996 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower771 (Follower): Commit index set to 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 0 (of expected -1)
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower771 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): Commit index set to 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower771 (Follower): Setting last applied to 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower771: Moving last applied index from -1 to 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower771: Applying state for log index 0 data zero
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower771: applyState called: zero
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 0 (of expected -1)
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 0
01:07:15,997 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to 0
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower770: Applying state for log index 0 data zero
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower770: applyState called: zero
01:07:15,997 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 255.6 μs
01:07:15,997 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:15,997 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:15,997 PM [follower771-writer-163] [DEBUG] RaftStorageCompleter - follower771: 1 completion(s) pending
01:07:15,997 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 6.212 ms
01:07:15,997 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 5.080 μs
01:07:15,997 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - follower771: running 1 completion(s)
01:07:15,997 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: completed 1 commands in 6.014 ms
01:07:15,997 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower771: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:15,997 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: received 1 commands after 6.530 μs
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:15,997 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:15,997 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 119.4 μs
01:07:15,997 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:15,997 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:15,997 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:15,997 PM [follower771-writer-163] [DEBUG] RaftStorageCompleter - follower771: 1 completion(s) pending
01:07:15,997 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - follower771: running 1 completion(s)
01:07:15,997 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: completed 1 commands in 216.1 μs
01:07:15,997 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: waiting to receive commands
01:07:15,997 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader769: Persist data index=1 term=6 command=one
01:07:15,997 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:15,997 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 1 is present for follower follower771
01:07:15,998 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 831.8 μs
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower771: AppendEntries{term=6, leaderId=leader769, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 1 is present for follower follower770
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Append entry to log one
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Log size is now 2
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,998 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 0 (of expected -1)
01:07:15,998 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:15,998 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 669.9 μs
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): Number of entries to be appended = 1
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): Append entry to log one
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower771: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): Log size is now 2
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower771, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:15,998 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: received 1 commands after 995.8 μs
01:07:15,998 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 0 (of expected -1)
01:07:16,0 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,0 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true}
01:07:16,0 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 2.649 ms
01:07:16,0 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,0 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 2.331 ms
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@4638dd94
01:07:16,0 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 1, nextIndex: 2
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
01:07:16,0 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 1
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 0 to 1
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader769: Applying state for log index 1 data one
01:07:16,0 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader769: applyState called: one
01:07:16,0 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader769: fakeSnapshot purging log to 0 for term 6
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:16,1 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 272.3 μs
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Commit index set to 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 0 to 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower770: Applying state for log index 1 data one
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower770: applyState called: one
01:07:16,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower771 updated: matchIndex: 1, nextIndex: 2
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower770: fakeSnapshot purging log to 0 for term 6
01:07:16,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:16,1 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 527.0 μs
01:07:16,1 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower771 (Follower): Commit index set to 1
01:07:16,1 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,1 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 290.0 μs
01:07:16,1 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,1 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,1 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 117.4 μs
01:07:16,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,1 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower771 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower771 (Follower): Setting last applied to 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower771: Moving last applied index from 0 to 1
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower771: Applying state for log index 1 data one
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower771: applyState called: one
01:07:16,1 PM [follower771-writer-163] [DEBUG] RaftStorageCompleter - follower771: 1 completion(s) pending
01:07:16,1 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: completed 1 commands in 2.746 ms
01:07:16,1 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: waiting to receive commands
01:07:16,1 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: received 1 commands after 80.65 μs
01:07:16,1 PM [follower771-writer-163] [DEBUG] RaftStorageCompleter - follower771: 1 more completion(s) pending
01:07:16,1 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: completed 1 commands in 84.64 μs
01:07:16,1 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: waiting to receive commands
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower771: fakeSnapshot purging log to 0 for term 6
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower771: running 2 completion(s)
01:07:16,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower771: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:16,219 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,220 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,340 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,459 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,650 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader769: Persist data index=2 term=6 command=two
01:07:16,650 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 
01:07:16,650 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 649.0 ms
01:07:16,651 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
01:07:16,651 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
01:07:16,651 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
01:07:16,651 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 2 is present for follower follower770
01:07:16,651 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:16,651 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Append entry to log two
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Log size is now 2
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,651 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 1 (of expected -1)
01:07:16,651 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 650.5 ms
01:07:16,654 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,654 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,654 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 2.852 ms
01:07:16,654 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,654 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 4.354 ms
01:07:16,654 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,654 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,654 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false}
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true}
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@432a9be0
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 2, nextIndex: 3
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 2
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 1 to 2
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader769: Applying state for log index 2 data two
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader769: applyState called: two
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader769: fakeSnapshot purging log to 1 for term 6
01:07:16,655 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 701.6 μs
01:07:16,655 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Follower - follower770 (Follower): Commit index set to 2
01:07:16,655 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 2 (of expected -1)
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
01:07:16,655 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 179.0 μs
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 2
01:07:16,655 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 1 to 2
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftActor - follower770: Applying state for log index 2 data two
01:07:16,655 PM [test-pekko.actor.default-dispatcher-13] [INFO] MockRaftActor - follower770: applyState called: two
01:07:16,656 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] SnapshotManager - follower770: fakeSnapshot purging log to 1 for term 6
01:07:16,656 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 1.107 ms
01:07:16,656 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,656 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 86.27 μs
01:07:16,656 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,656 PM [test-pekko.actor.default-dispatcher-13] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,656 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Persist data index=3 term=6 command=three
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true
01:07:16,656 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 616.0 μs
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 3 is present for follower follower770
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,656 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:16,656 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,657 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0
01:07:16,657 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Append entry to log three
01:07:16,657 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 
01:07:16,657 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Log size is now 2
01:07:16,657 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,657 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 2 (of expected -1)
01:07:16,657 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 953.2 μs
01:07:16,658 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,658 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 2.093 ms
01:07:16,658 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,658 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,658 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true}
01:07:16,658 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false]
01:07:16,658 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader769: captured snapshot at lastSequenceNumber: 4
01:07:16,658 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: Persising snapshot at EntryInfo[index=2, term=6]/EntryInfo[index=3, term=6]
01:07:16,658 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader769: support() called
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 6
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@1fd3d26c
01:07:16,659 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.tmp
01:07:16,659 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 3, nextIndex: 4
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 3
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 2 to 3
01:07:16,659 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 2.487 ms
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Applying state for log index 3 data three
01:07:16,659 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader769: applyState called: three
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:16,659 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 1.154 ms
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Persist data index=4 term=6 command=four
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true
01:07:16,659 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:16,659 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 147.1 μs
01:07:16,659 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 4 is present for follower follower770
01:07:16,659 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 3.770 μs
01:07:16,660 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=3, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=6, command=four, pending=true}]}
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,660 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false}
01:07:16,660 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Commit index set to 3
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 3 (of expected -1)
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 3
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 2 to 3
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower770: Applying state for log index 3 data three
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower770: applyState called: three
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=3, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=6, command=four, pending=true}]}
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 3, entries to be added from: 0
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Append entry to log four
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Log size is now 3
01:07:16,660 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 759.5 μs
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=4, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 3 (of expected -1)
01:07:16,661 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,661 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 614.8 μs
01:07:16,661 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 3.360 μs
01:07:16,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,661 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,661 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,661 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=true}
01:07:16,661 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 1.891 ms
01:07:16,661 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,661 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@4e98c3bc
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 4 is present for follower follower770
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 4, nextIndex: 5
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 3 to 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Applying state for log index 4 data four
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader769: applyState called: four
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1
01:07:16,662 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 329.6 μs
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:16,662 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,662 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 97.24 μs
01:07:16,662 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorage - leader769: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:16.659385008Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1, size=510, entryCompress=NONE, stateCompress=NONE}
01:07:16,662 PM [leader769-%d494162] [DEBUG] RaftStorageCompleter - leader769: 1 more completion(s) pending
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 2 completion(s)
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: snapshot is durable as of 2025-07-01T13:07:16.659385008Z
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader769: Snapshot success -  sequence number: 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Commit index set to 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 4 (of expected -1)
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 3 to 4
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower770: Applying state for log index 4 data four
01:07:16,662 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower770: applyState called: four
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorSnapshotMessageSupport - leader769: SnapshotComplete received
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Persist data index=5 term=6 command=five
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:16,662 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 2 commands after 419.8 μs
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 5 is present for follower follower770
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=4, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:16,662 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=4, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 4, entries to be added from: 0
01:07:16,663 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Append entry to log five
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 
01:07:16,663 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 2.042 ms
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Log size is now 4
01:07:16,663 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 2 commands after 15.99 μs
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=5, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 4 (of expected -1)
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,663 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false}
01:07:16,664 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 2 completion(s) pending
01:07:16,664 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 2 completion(s) pending
01:07:16,664 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 2 commands in 1.131 ms
01:07:16,664 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,664 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 2 commands in 1.520 ms
01:07:16,664 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 2 completion(s)
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=true}
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@62424ef5
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower770: running 2 completion(s)
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 5, nextIndex: 6
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false}
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 5
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 4 to 5
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Applying state for log index 5 data five
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader769: applyState called: five
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:16,664 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 253.9 μs
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Persist data index=6 term=6 command=six
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): Commit index set to 5
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 5 (of expected -1)
01:07:16,664 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 5
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 4 to 5
01:07:16,664 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 89.33 μs
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower770: Applying state for log index 5 data five
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 1
01:07:16,664 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower770: applyState called: five
01:07:16,664 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 2.220 μs
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 1
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 6 is present for follower follower770
01:07:16,664 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 512.1 μs
01:07:16,664 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,665 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 83.35 μs
01:07:16,665 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,664 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=5, prevLogTerm=6, leaderCommit=5, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]}
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=5, prevLogTerm=6, leaderCommit=5, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]}
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 5, entries to be added from: 0
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): Append entry to log six
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): Log size is now 5
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,665 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 5 (of expected -1)
01:07:16,665 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 279.1 μs
01:07:16,665 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,665 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 568.0 μs
01:07:16,665 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=true}
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@4ad67995
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 6 is present for follower follower770
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 6, nextIndex: 7
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 6
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 5 to 6
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Applying state for log index 6 data six
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader769: applyState called: six
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1
01:07:16,665 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 311.2 μs
01:07:16,665 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): Commit index set to 6
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 6 (of expected -1)
01:07:16,665 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 100.1 μs
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6
01:07:16,665 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 6
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 5 to 6
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower770: Applying state for log index 6 data six
01:07:16,665 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower770: applyState called: six
01:07:16,665 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,666 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,666 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 1.215 ms
01:07:16,666 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,666 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 3.420 μs
01:07:16,666 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false}
01:07:16,666 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,666 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 87.50 μs
01:07:16,666 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,666 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,751 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1
01:07:16,752 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties
01:07:16,752 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:16,752 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:16,752 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: picked SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:16.659385008Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1, size=510, entryCompress=NONE, stateCompress=NONE} as the latest file
01:07:16,752 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaInstallSnapshot: sending 1 more payload to trigger second snapshot
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader769: Persist data index=7 term=6 command=seven
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=7, term=6, command=seven, pending=true} 
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=seven}, logIndex: 7, isSendImmediate: true
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 7, leaderSnapShotIndex: 1
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 1
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 7 is present for follower follower770
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=6, prevLogTerm=6, leaderCommit=6, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=6, command=seven, pending=true}]}
01:07:16,753 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,753 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 87.72 ms
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=6, prevLogTerm=6, leaderCommit=6, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=6, command=seven, pending=true}]}
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower770 (Follower): Append entry to log seven
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=7, term=6, command=seven, pending=true} 
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower770 (Follower): Log size is now 6
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=7, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,754 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 6 (of expected -1)
01:07:16,754 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 87.54 ms
01:07:16,755 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,755 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,755 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 1.593 ms
01:07:16,755 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 2.228 ms
01:07:16,755 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,755 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,755 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,755 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=7, term=6, command=seven, pending=false}
01:07:16,755 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,755 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=7, term=6, command=seven, pending=true}
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader769: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=6, lastAppliedTerm=6, lastIndex=7, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false]
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader769: captured snapshot at lastSequenceNumber: 8
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader769: Persising snapshot at EntryInfo[index=6, term=6]/EntryInfo[index=7, term=6]
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader769: support() called
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader769: log size 6 exceeds the snapshot batch count 4 - doing snapshotPreCommit with index 6
01:07:16,756 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.tmp
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader769: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 6 and term: 6
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@3fe0201e
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 7, nextIndex: 8
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 7
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 6 to 7
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader769: Applying state for log index 7 data seven
01:07:16,756 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader769: applyState called: seven
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6
01:07:16,757 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 1.227 ms
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Commit index set to 7
01:07:16,757 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 7 (of expected -1)
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7
01:07:16,757 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 220.9 μs
01:07:16,757 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 7
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 6 to 7
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower770: Applying state for log index 7 data seven
01:07:16,757 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - follower770: applyState called: seven
01:07:16,757 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,757 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 1.744 ms
01:07:16,757 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,757 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 296.8 μs
01:07:16,757 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,757 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,758 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1
01:07:16,758 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1
01:07:16,758 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1
01:07:16,759 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties
01:07:16,759 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:16,759 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:16,759 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: deleted snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:16.659385008Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-274D6AB0.v1, size=510, entryCompress=NONE, stateCompress=NONE}
01:07:16,759 PM [leader769-%d494163] [DEBUG] RaftStorage - leader769: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=6, term=6], timestamp=2025-07-01T13:07:16.756603251Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1, size=578, entryCompress=NONE, stateCompress=NONE}
01:07:16,759 PM [leader769-%d494163] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,759 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,759 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader769: snapshot is durable as of 2025-07-01T13:07:16.756603251Z
01:07:16,759 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader769: Snapshot success -  sequence number: 8
01:07:16,759 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorSnapshotMessageSupport - leader769: SnapshotComplete received
01:07:16,759 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 2.005 ms
01:07:16,759 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,759 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 118.6 μs
01:07:16,759 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,759 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,939 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 7 (of expected -1)
01:07:16,955 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1
01:07:16,955 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties
01:07:16,955 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:16,955 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:16,955 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: picked SnapshotFileV1{lastIncluded=EntryInfo[index=6, term=6], timestamp=2025-07-01T13:07:16.756603251Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1, size=578, entryCompress=NONE, stateCompress=NONE} as the latest file
01:07:16,957 PM [main] [DEBUG] PeerInfos - leader769: Updated server config: isVoting: true, peers: [PeerInfo [id=follower771, address=pekko://test/user/follower771, votingState=NON_VOTING], PeerInfo [id=follower770, address=pekko://test/user/follower770, votingState=NON_VOTING]]
01:07:16,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Persist data index=8 term=6 command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}
01:07:16,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: Append log entry and persist JournaledLogEntry{index=8, term=6, command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}, pending=true} 
01:07:16,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Replicate message: identifier: MockIdentifier{value=serverConfig}, logIndex: 8, isSendImmediate: true
01:07:16,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower771: active: false, followerNextIndex: 2, leaderLastIndex: 8, leaderSnapShotIndex: 6
01:07:16,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6
01:07:16,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendAppendEntries: 8 is present for follower follower770
01:07:16,958 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader769, prevLogIndex=7, prevLogTerm=6, leaderCommit=7, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=6, command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}, pending=true}]}
01:07:16,958 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: awaiting more completions to resolve 1 synchronized callback(s)
01:07:16,958 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 198.6 ms
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader769, prevLogIndex=7, prevLogTerm=6, leaderCommit=7, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=6, command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}, pending=true}]}
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Append entry to log VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=8, term=6, command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}, pending=true} 
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PeerInfos - follower770: Updated server config: isVoting: false, peers: [PeerInfo [id=leader769, address=pekko://test/user/leader769, votingState=VOTING], PeerInfo [id=follower771, address=pekko://test/user/follower771, votingState=NON_VOTING]]
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): Log size is now 7
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=8, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:16,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 7 (of expected -1)
01:07:16,958 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 200.7 ms
01:07:16,960 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,960 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 2.620 ms
01:07:16,960 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,960 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,960 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,960 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 2.212 ms
01:07:16,960 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:16,960 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader769: persist complete JournaledLogEntry{index=8, term=6, command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}, pending=true}
01:07:16,960 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=8, term=6, command=VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}, pending=false}
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader769 (Leader): Setting last applied to 8
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader769: Moving last applied index from 7 to 8
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader769: Applying state for log index 8 data VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@7f45294f
01:07:16,961 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 296.6 μs
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 8, nextIndex: 9
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 6
01:07:16,961 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:16,961 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:16,961 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 122.3 μs
01:07:16,961 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): Commit index set to 8
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 8
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 8
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 7 to 8
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower770: Applying state for log index 8 data VotingConfig{serverInfo=[ServerInfo[peerId=leader769, isVoting=true], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=follower771, isVoting=false]]}
01:07:16,961 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 700.7 μs
01:07:16,961 PM [follower770-writer-162] [DEBUG] RaftStorageCompleter - follower770: 1 completion(s) pending
01:07:16,961 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 104.6 μs
01:07:16,961 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower770: running 1 completion(s)
01:07:16,961 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: waiting to receive commands
01:07:17,9 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - verifyInstallSnapshotToLaggingFollower starting
01:07:17,60 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower771: Lagging -2 entries behind leader leader769 and reached 1 (of expected -1)
01:07:17,60 PM [test-pekko.actor.default-dispatcher-18] [WARN] AbstractLeader - leader769 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=6, success=true, followerId=follower771, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 839, lastApplied : 8, commitIndex : 8
01:07:17,60 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractLeader - leader769 (Leader): Initiating install snapshot to follower follower771: follower nextIndex: 2, leader snapshotIndex: 6, leader lastIndex: 8, leader log size: 2
01:07:17,60 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader769: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=8, lastAppliedTerm=6, lastIndex=8, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower771
01:07:17,60 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader769: lastSequenceNumber prior to capture: 10
01:07:17,60 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader769: support() called
01:07:17,61 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower771: Lagging -2 entries behind leader leader769 and reached 1 (of expected -1)
01:07:17,61 PM [leader769-%d494164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:17,61 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:17,61 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: Persising snapshot at EntryInfo[index=8, term=6]/EntryInfo[index=8, term=6]
01:07:17,61 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader769: support() called
01:07:17,62 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 6 and term: 6
01:07:17,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): sendInstallSnapshot
01:07:17,62 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD85-03B06D69.tmp
01:07:17,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] LeaderInstallSnapshotState - leader769 (Leader): Snapshot 396 bytes, total chunks to send: 1
01:07:17,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] LeaderInstallSnapshotState - leader769 (Leader): Next chunk: total length=396, offset=0, size=396, hashCode=-153134944
01:07:17,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): next snapshot chunk size for follower follower771: 396
01:07:17,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): InstallSnapshot sent to follower Vector(user, follower771), Chunk: 1/1
01:07:17,62 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): handleInstallSnapshot: InstallSnapshot{term=6, leaderId=leader769, lastIncludedIndex=8, lastIncludedTerm=6, datasize=396, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=follower771, isVoting=false], ServerInfo[peerId=follower770, isVoting=false], ServerInfo[peerId=leader769, isVoting=true]]}}
01:07:17,62 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower771: Lagging 7 entries behind leader leader769 and reached 1 (of expected -1)
01:07:17,62 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotTracker - follower771 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
01:07:17,62 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower771 (Follower): Snapshot received from leader: leader769
01:07:17,62 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - follower771: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=396}}
01:07:17,62 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower771: support() called
01:07:17,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower771: Converted InstallSnapshot from leader: leader769 to state
01:07:17,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower771: lastSequenceNumber prior to persisting applied snapshot: 3
01:07:17,63 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower771: support() called
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD85-03B06D69.v1
01:07:17,64 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/snapshot-000000006863DD85-03D13302.tmp
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD85-03B06D69.v1
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: deleted snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=6, term=6], timestamp=2025-07-01T13:07:16.756603251Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD84-2D18D973.v1, size=578, entryCompress=NONE, stateCompress=NONE}
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorage - leader769: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=8, term=6], timestamp=2025-07-01T13:07:17.061893993Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD85-03B06D69.v1, size=500, entryCompress=NONE, stateCompress=NONE}
01:07:17,64 PM [leader769-%d494165] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:17,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:17,64 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader769: snapshot is durable as of 2025-07-01T13:07:17.061893993Z
01:07:17,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader769: Snapshot success -  sequence number: 10
01:07:17,65 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorSnapshotMessageSupport - leader769: SnapshotComplete received
01:07:17,65 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 103.7 ms
01:07:17,65 PM [leader769-writer-164] [DEBUG] RaftStorageCompleter - leader769: 1 completion(s) pending
01:07:17,65 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 166.7 μs
01:07:17,65 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: waiting to receive commands
01:07:17,65 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader769: running 1 completion(s)
01:07:17,66 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/snapshot-000000006863DD85-03D13302.v1
01:07:17,66 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/snapshot-000000006863DD85-03D13302.v1
01:07:17,67 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/TermInfo.properties
01:07:17,67 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/journal-v1-1.log
01:07:17,67 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/metadata-v1-1.log
01:07:17,67 PM [follower771-%d646871] [DEBUG] RaftStorage - follower771: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=8, term=6], timestamp=2025-07-01T13:07:17.064041730Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/follower771/snapshot-000000006863DD85-03D13302.v1, size=500, entryCompress=NONE, stateCompress=NONE}
01:07:17,67 PM [follower771-%d646871] [DEBUG] RaftStorageCompleter - follower771: 1 completion(s) pending
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower771: running 1 completion(s)
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - follower771: snapshot is durable as of 2025-07-01T13:07:17.064041730Z
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower771: Snapshot success -  sequence number: 3
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PeerInfos - follower771: Updated server config: isVoting: false, peers: [PeerInfo [id=leader769, address=pekko://test/user/leader769, votingState=VOTING], PeerInfo [id=follower770, address=pekko://test/user/follower770, votingState=NON_VOTING]]
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower771: support() called
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower771: applySnapshot called
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower771 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=6, followerId=follower771, chunkIndex=1, success=true}
01:07:17,67 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorSnapshotMessageSupport - follower771: SnapshotComplete received
01:07:17,67 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader769 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=6, followerId=follower771, chunkIndex=1, success=true}
01:07:17,67 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractLeader - leader769 (Leader): Snapshot successfully installed on follower follower771 (last chunk 1) - matchIndex set to 8, nextIndex set to 9
01:07:17,67 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader769: fakeSnapshot purging log to 7 for term 6
01:07:17,67 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: received 1 commands after 1.066 s
01:07:17,68 PM [follower771-writer-163] [DEBUG] RaftStorageCompleter - follower771: 1 completion(s) pending
01:07:17,68 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: completed 1 commands in 214.5 μs
01:07:17,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower771: running 1 completion(s)
01:07:17,68 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: waiting to receive commands
01:07:17,180 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:17,180 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower770: fakeSnapshot purging log to 7 for term 6
01:07:17,180 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:17,299 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:17,299 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:17,313 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD85-03B06D69.v1
01:07:17,313 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/TermInfo.properties
01:07:17,313 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/journal-v1-1.log
01:07:17,313 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/metadata-v1-1.log
01:07:17,313 PM [awaitility-thread] [DEBUG] RaftStorage - leader769: picked SnapshotFileV1{lastIncluded=EntryInfo[index=8, term=6], timestamp=2025-07-01T13:07:17.061893993Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest3002278042366879094/leader769/snapshot-000000006863DD85-03B06D69.v1, size=500, entryCompress=NONE, stateCompress=NONE} as the latest file
01:07:17,419 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower771: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:17,419 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader769 and reached 8 (of expected -1)
01:07:17,469 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - verifyInstallSnapshotToLaggingFollower complete
01:07:17,469 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaInstallSnapshot complete
01:07:17,469 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower770-collector772#1801634279]
01:07:17,480 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower770]
01:07:17,480 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower770: terminating thread VirtualThread[#931,follower770-writer-162]/waiting
01:07:17,480 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: received 1 commands after 518.9 ms
01:07:17,480 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: completed 1 commands in 82.21 μs
01:07:17,480 PM [follower770-writer-162] [DEBUG] JournalWriteTask - follower770: journal writer stopped
01:07:17,481 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower770: journal closed
01:07:17,481 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower770: journal closed
01:07:17,481 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks
01:07:17,481 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks
01:07:17,481 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower771-collector773#-1990777440]
01:07:17,481 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower771]
01:07:17,481 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower771: terminating thread VirtualThread[#932,follower771-writer-163]/waiting
01:07:17,481 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: received 1 commands after 413.7 ms
01:07:17,481 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: completed 1 commands in 12.96 μs
01:07:17,482 PM [follower771-writer-163] [DEBUG] JournalWriteTask - follower771: journal writer stopped
01:07:17,482 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower771: journal closed
01:07:17,482 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower771: journal closed
01:07:17,482 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower771: stopped executor with 0 remaining tasks
01:07:17,482 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower771: stopped executor with 0 remaining tasks
01:07:17,482 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader769-collector774#1236448986]
01:07:17,482 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader769]
01:07:17,482 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - leader769: terminating thread VirtualThread[#933,leader769-writer-164]/waiting
01:07:17,482 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: received 1 commands after 417.5 ms
01:07:17,482 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: completed 1 commands in 12.87 μs
01:07:17,482 PM [leader769-writer-164] [DEBUG] JournalWriteTask - leader769: journal writer stopped
01:07:17,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader769: journal closed
01:07:17,483 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader769: journal closed
01:07:17,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader769: stopped executor with 0 remaining tasks
01:07:17,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader769: stopped executor with 0 remaining tasks
01:07:17,485 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaAppendEntries starting
[WARN] [07/01/2025 13:07:17.487] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower779]: org.opendaylight.controller.cluster.common.actor.Monitor@18c3221b
01:07:17,488 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower779: Starting recovery with journal batch size 1000
01:07:17,488 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower779: started executor
01:07:17,488 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower779: started executor
01:07:17,488 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower779: metadata journal open: replayFrom=1 applyTo=0
01:07:17,489 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower779: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:17,489 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower779: journal open: applyTo=0
01:07:17,489 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: journal writer started
01:07:17,489 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
[WARN] [07/01/2025 13:07:17.490] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower780]: org.opendaylight.controller.cluster.common.actor.Monitor@52422e8f
01:07:17,490 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/follower779/TermInfo.properties does not exist
01:07:17,490 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/follower779/journal-v1-1.log
01:07:17,490 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/follower779/metadata-v1-1.log
01:07:17,490 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower779: no eligible files found
01:07:17,491 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - follower779: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:17,491 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower780: Starting recovery with journal batch size 1000
01:07:17,491 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower780: started executor
01:07:17,491 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower780: started executor
01:07:17,491 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower780: metadata journal open: replayFrom=1 applyTo=0
01:07:17,493 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower780: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:17,493 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower780: journal open: applyTo=0
01:07:17,493 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: journal writer started
01:07:17,493 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: waiting to receive commands
01:07:17,493 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=0, votedFor=null
01:07:17,493 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - follower779: Local TermInfo store seeded with TermInfo{term=0}
01:07:17,493 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower779: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1a42e485
01:07:17,493 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to -1
01:07:17,494 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - follower779: journal recovery completedin 276.1 μs with journalIndex=1
01:07:17,494 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/follower780/TermInfo.properties does not exist
01:07:17,494 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower780: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/follower780/journal-v1-1.log
01:07:17,494 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower780: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/follower780/metadata-v1-1.log
01:07:17,494 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower780: no eligible files found
01:07:17,495 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower780: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
[WARN] [07/01/2025 13:07:17.495] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader778]: org.opendaylight.controller.cluster.common.actor.Monitor@223907b0
01:07:17,495 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - leader778: Starting recovery with journal batch size 1000
01:07:17,495 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader778: started executor
01:07:17,495 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader778: started executor
01:07:17,496 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader778: metadata journal open: replayFrom=1 applyTo=0
01:07:17,496 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader778: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:17,496 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - leader778: journal open: applyTo=0
01:07:17,497 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: journal writer started
01:07:17,497 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:17,498 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/TermInfo.properties does not exist
01:07:17,498 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/journal-v1-1.log
01:07:17,498 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/metadata-v1-1.log
01:07:17,498 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader778: no eligible files found
01:07:17,498 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader778: Set currentTerm=5, votedFor=leader778
01:07:17,498 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader778: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:17,499 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower780: Set currentTerm=0, votedFor=null
01:07:17,499 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower780: Local TermInfo store seeded with TermInfo{term=0}
01:07:17,499 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower780: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@5a385f55
01:07:17,499 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from -1 to -1
01:07:17,499 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower780: journal recovery completedin 224.0 μs with journalIndex=1
01:07:17,501 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader778: Set currentTerm=5, votedFor=leader778
01:07:17,501 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader778: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader778}
01:07:17,501 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader778: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@713d840c
01:07:17,501 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from -1 to -1
01:07:17,501 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader778: journal recovery completedin 216.0 μs with journalIndex=1
01:07:17,999 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
01:07:17,999 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader778 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.139:17355, Removed)
01:07:17,999 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader778 (Follower): Received ElectionTimeout - switching to Candidate
01:07:17,999 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader778 (Candidate): Election: Candidate has following voting peers: [follower779, follower780]
01:07:18,3 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader778: Set currentTerm=6, votedFor=leader778
01:07:18,3 PM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader778 (Candidate): Starting new election term 6
01:07:18,3 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader778 (Candidate): Sending RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1} to peer follower779
01:07:18,3 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader778 (Candidate): Sending RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1} to peer follower780
01:07:18,3 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader778 (Follower) :- Switching from behavior Follower to Candidate, election term: 6
01:07:18,4 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:18,4 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower780 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:18,4 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: []
01:07:18,4 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower780 (Follower): Cluster state: []
01:07:18,4 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:18,4 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower780 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:18,4 PM [test-pekko.actor.default-dispatcher-7] [INFO] Follower - follower779 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:18,4 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower780 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:18,6 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower780: Set currentTerm=6, votedFor=null
01:07:18,6 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=null
01:07:18,6 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower780 (Follower): In requestVote: RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:18,6 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=6, candidateId=leader778, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:18,9 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower780: Set currentTerm=6, votedFor=leader778
01:07:18,9 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower780 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:18,9 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=leader778
01:07:18,9 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:18,9 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader778 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6
01:07:18,9 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader778 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1
01:07:18,9 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): Election: Leader has following peers: [follower779, follower780]
01:07:18,9 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:18,9 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:18,9 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader778 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6
01:07:18,9 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower780: Last sync leader does not match current leader leader778, need to catch up to -1
01:07:18,10 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower780 updated: matchIndex: -1, nextIndex: 0
01:07:18,10 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:18,10 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower779: Last sync leader does not match current leader leader778, need to catch up to -1
01:07:18,10 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: -1, nextIndex: 0
01:07:18,10 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [07/01/2025 13:07:18.010] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower780] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower780, initialSyncDone=false]
[WARN] [07/01/2025 13:07:18.010] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=false]
01:07:18,54 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected
01:07:18,54 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader778: Persist data index=0 term=6 command=zero
01:07:18,54 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader778: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:18,54 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 557.7 ms
01:07:18,55 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
01:07:18,55 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:18,55 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 0 is present for follower follower779
01:07:18,55 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader778, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Append entry to log zero
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Log size is now 1
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,55 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached -1 (of expected -1)
01:07:18,55 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 566.1 ms
[WARN] [07/01/2025 13:07:18.055] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=true]
01:07:18,56 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:18,56 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 0 is present for follower follower780
01:07:18,56 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): Number of entries to be appended = 1
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): Append entry to log zero
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower780: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): Log size is now 1
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower780, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,56 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached -1 (of expected -1)
01:07:18,56 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 1 commands after 562.9 ms
[WARN] [07/01/2025 13:07:18.056] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower780] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower780, initialSyncDone=true]
01:07:18,56 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: awaiting more completions to resolve 1 synchronized callback(s)
01:07:18,61 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,61 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 7.074 ms
01:07:18,61 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader778: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true}
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@68988774
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 0, nextIndex: 1
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader778 (Leader): Setting last applied to 0
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from -1 to 0
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader778: Applying state for log index 0 data zero
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader778: applyState called: zero
01:07:18,62 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 521.5 μs
01:07:18,62 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,62 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 135.3 μs
01:07:18,62 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Commit index set to 0
01:07:18,62 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower780 updated: matchIndex: 0, nextIndex: 1
01:07:18,63 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 0 (of expected -1)
01:07:18,63 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:18,63 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower779: Applying state for log index 0 data zero
01:07:18,63 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower779: applyState called: zero
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): Commit index set to 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 0 (of expected -1)
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower780 (Follower): Setting last applied to 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from -1 to 0
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower780: Applying state for log index 0 data zero
01:07:18,63 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower780: applyState called: zero
01:07:18,63 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader778: Persist data index=1 term=6 command=one
01:07:18,63 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader778: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:18,63 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,63 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 7.922 ms
01:07:18,63 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 1.061 ms
01:07:18,63 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 5.560 μs
01:07:18,63 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 more completion(s) pending
01:07:18,63 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 139.6 μs
01:07:18,63 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
01:07:18,64 PM [follower780-writer-166] [DEBUG] RaftStorageCompleter - follower780: 1 completion(s) pending
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower780: running 1 completion(s)
01:07:18,64 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 1 commands in 7.671 ms
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower780: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:18,64 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 1 commands after 5.370 μs
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower779: running 2 completion(s)
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 1 is present for follower follower779
01:07:18,64 PM [follower780-writer-166] [DEBUG] RaftStorageCompleter - follower780: 1 completion(s) pending
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower780: running 1 completion(s)
01:07:18,64 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 1 commands in 146.8 μs
01:07:18,64 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: waiting to receive commands
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader778, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 1 is present for follower follower780
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: awaiting more completions to resolve 1 synchronized callback(s)
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Append entry to log one
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower780 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower780 (Follower): Number of entries to be appended = 1
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower780 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Log size is now 2
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower780 (Follower): Append entry to log one
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower780: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 0 (of expected -1)
01:07:18,64 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 742.7 μs
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower780 (Follower): Log size is now 2
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower780 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower780, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,64 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 0 (of expected -1)
01:07:18,64 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 1 commands after 482.0 μs
01:07:18,65 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,65 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 1.624 ms
01:07:18,65 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,65 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,65 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader778: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true}
01:07:18,65 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader778: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@1ffa56d4
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 1, nextIndex: 2
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader778 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader778 (Leader): Setting last applied to 1
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from 0 to 1
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader778: Applying state for log index 1 data one
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader778: applyState called: one
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader778: fakeSnapshot purging log to 0 for term 6
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower780 updated: matchIndex: 1, nextIndex: 2
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): Commit index set to 1
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 1 (of expected -1)
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower780 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower780 (Follower): Setting last applied to 1
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from 0 to 1
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower780: Applying state for log index 1 data one
01:07:18,65 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower780: applyState called: one
01:07:18,66 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower780: fakeSnapshot purging log to 0 for term 6
01:07:18,66 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 696.1 μs
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): Commit index set to 1
01:07:18,66 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,66 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 92.72 μs
01:07:18,66 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,66 PM [follower780-writer-166] [DEBUG] RaftStorageCompleter - follower780: 1 completion(s) pending
01:07:18,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,66 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 1 commands in 1.505 ms
01:07:18,66 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorageCompleter - follower780: running 1 completion(s)
01:07:18,66 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 1 commands after 2.710 μs
01:07:18,66 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower780: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:18,66 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 1 (of expected -1)
01:07:18,66 PM [follower780-writer-166] [DEBUG] RaftStorageCompleter - follower780: 1 completion(s) pending
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:18,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower780: running 1 completion(s)
01:07:18,66 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 1.663 ms
01:07:18,66 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,66 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 1 commands in 70.26 μs
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 1
01:07:18,66 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: waiting to receive commands
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 0 to 1
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower779: Applying state for log index 1 data one
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower779: applyState called: one
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower779: fakeSnapshot purging log to 0 for term 6
01:07:18,66 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 257.3 μs
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,66 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,66 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:18,66 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 78.93 μs
01:07:18,67 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,67 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,249 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 1 (of expected -1)
01:07:18,249 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 1 (of expected -1)
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader778: Persist data index=2 term=6 command=two
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader778: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower779
01:07:18,257 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 191.0 ms
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower780
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:18,257 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: awaiting more completions to resolve 1 synchronized callback(s)
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): Append entry to log two
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): Log size is now 2
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,257 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 190.7 ms
01:07:18,257 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 1 (of expected -1)
01:07:18,259 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,259 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,259 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 1.753 ms
01:07:18,259 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,259 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 2.186 ms
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,259 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,259 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader778: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true}
01:07:18,259 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false}
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@73c34dac
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 2, nextIndex: 3
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader778 (Leader): Setting last applied to 2
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from 1 to 2
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader778: Applying state for log index 2 data two
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader778: applyState called: two
01:07:18,259 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader778: fakeSnapshot purging log to 1 for term 6
01:07:18,260 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
01:07:18,260 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 446.9 μs
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): Commit index set to 2
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 2 (of expected -1)
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
01:07:18,260 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 2
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 1 to 2
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower779: Applying state for log index 2 data two
01:07:18,260 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 157.3 μs
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower779: applyState called: two
01:07:18,260 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,260 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower779: fakeSnapshot purging log to 1 for term 6
01:07:18,260 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 909.5 μs
01:07:18,260 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,260 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 108.0 μs
01:07:18,260 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,260 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader778: Persist data index=3 term=6 command=three
01:07:18,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader778: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 
01:07:18,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true
01:07:18,307 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:18,308 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 3 is present for follower follower779
01:07:18,308 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader778, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:18,308 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 47.79 ms
01:07:18,308 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:18,308 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower780
01:07:18,308 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}, JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:18,308 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader778: awaiting more completions to resolve 1 synchronized callback(s)
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): Append entry to log three
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): Log size is now 2
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,308 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 47.90 ms
01:07:18,308 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 2 (of expected -1)
01:07:18,310 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,310 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,310 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 1.770 ms
01:07:18,310 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 2.216 ms
01:07:18,310 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,310 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,310 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,310 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,310 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader778: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true}
01:07:18,310 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false}
01:07:18,310 PM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader778: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false]
01:07:18,310 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader778: captured snapshot at lastSequenceNumber: 4
01:07:18,310 PM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader778: Persising snapshot at EntryInfo[index=2, term=6]/EntryInfo[index=3, term=6]
01:07:18,310 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader778: support() called
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader778: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 6
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader778: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@62dfeb95
01:07:18,311 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/snapshot-000000006863DD86-12893FD0.tmp
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 3, nextIndex: 4
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader778 (Leader): Setting last applied to 3
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from 2 to 3
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader778: Applying state for log index 3 data three
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader778: applyState called: three
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:18,311 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 1.087 ms
01:07:18,311 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): Commit index set to 3
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 3 (of expected -1)
01:07:18,311 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 146.1 μs
01:07:18,311 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3
01:07:18,311 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 3
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 2 to 3
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower779: Applying state for log index 3 data three
01:07:18,311 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower779: applyState called: three
01:07:18,311 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 1.543 ms
01:07:18,311 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,312 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 108.2 μs
01:07:18,312 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,312 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/snapshot-000000006863DD86-12893FD0.v1
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/snapshot-000000006863DD86-12893FD0.v1
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/TermInfo.properties
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/journal-v1-1.log
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/metadata-v1-1.log
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorage - leader778: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:18.310984656Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/snapshot-000000006863DD86-12893FD0.v1, size=510, entryCompress=NONE, stateCompress=NONE}
01:07:18,313 PM [leader778-%d343802] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,314 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,314 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader778: snapshot is durable as of 2025-07-01T13:07:18.310984656Z
01:07:18,314 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader778: Snapshot success -  sequence number: 4
01:07:18,314 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorSnapshotMessageSupport - leader778: SnapshotComplete received
01:07:18,314 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 2.521 ms
01:07:18,314 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,314 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 176.7 μs
01:07:18,314 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,314 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,370 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower780
01:07:18,370 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}, JournaledLogEntry{index=3, term=6, command=three, pending=false}]}
01:07:18,409 PM [awaitility-thread] [DEBUG] RaftStorage - leader778: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/snapshot-000000006863DD86-12893FD0.v1
01:07:18,409 PM [awaitility-thread] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/TermInfo.properties
01:07:18,409 PM [awaitility-thread] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/journal-v1-1.log
01:07:18,409 PM [awaitility-thread] [DEBUG] RaftStorage - leader778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/metadata-v1-1.log
01:07:18,409 PM [awaitility-thread] [DEBUG] RaftStorage - leader778: picked SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:18.310984656Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12976976649321426376/leader778/snapshot-000000006863DD86-12893FD0.v1, size=510, entryCompress=NONE, stateCompress=NONE} as the latest file
01:07:18,409 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaAppendEntries: sending 2 more payloads
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader778: Persist data index=4 term=6 command=four
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader778: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true
01:07:18,410 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 95.80 ms
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 4 is present for follower follower779
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader778, prevLogIndex=3, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=6, command=four, pending=true}]}
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower780
01:07:18,410 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: awaiting more completions to resolve 1 synchronized callback(s)
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=3, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=6, command=four, pending=true}]}
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 3, entries to be added from: 0
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): Append entry to log four
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): Log size is now 3
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=4, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 3 (of expected -1)
01:07:18,410 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 98.83 ms
01:07:18,412 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,412 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 1.947 ms
01:07:18,412 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader778: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=true}
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@3ea0eefc
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 4, nextIndex: 5
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader778 (Leader): Setting last applied to 4
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from 3 to 4
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader778: Applying state for log index 4 data four
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader778: applyState called: four
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:18,412 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 533.5 μs
01:07:18,412 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader778: Persist data index=5 term=6 command=five
01:07:18,412 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader778: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 
01:07:18,413 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 157.0 μs
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true
01:07:18,413 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 3.430 μs
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): Commit index set to 4
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 4 (of expected -1)
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 5 is present for follower follower779
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 4
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader778, prevLogIndex=4, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 3 to 4
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower779: Applying state for log index 4 data four
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower779: applyState called: four
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower780
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}, JournaledLogEntry{index=3, term=6, command=three, pending=false}, JournaledLogEntry{index=4, term=6, command=four, pending=false}, JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: awaiting more completions to resolve 1 synchronized callback(s)
01:07:18,413 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=4, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1
01:07:18,413 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 2.543 ms
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 4, entries to be added from: 0
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): Append entry to log five
01:07:18,413 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 5.310 μs
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): Log size is now 4
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=5, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 4 (of expected -1)
01:07:18,413 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 more completion(s) pending
01:07:18,413 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 126.0 μs
01:07:18,413 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 3.160 μs
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower779: running 2 completion(s)
01:07:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false}
01:07:18,413 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,413 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 900.6 μs
01:07:18,413 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,413 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader778: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=true}
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@6beeb62d
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 5 is present for follower follower779
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 5, nextIndex: 6
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader778 (Leader): Setting last applied to 5
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader778: Moving last applied index from 4 to 5
01:07:18,414 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,414 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 821.9 μs
01:07:18,414 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,414 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,414 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false}
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader778: Applying state for log index 5 data five
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader778: applyState called: five
01:07:18,414 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 732.8 μs
01:07:18,414 PM [leader778-writer-167] [DEBUG] RaftStorageCompleter - leader778: 1 completion(s) pending
01:07:18,414 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 112.1 μs
01:07:18,414 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: waiting to receive commands
01:07:18,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:18,415 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader778: running 1 completion(s)
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): Commit index set to 5
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 5
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 4 to 5
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower779: Applying state for log index 5 data five
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower779: applyState called: five
01:07:18,415 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 797.1 μs
01:07:18,415 PM [follower779-writer-165] [DEBUG] RaftStorageCompleter - follower779: 1 completion(s) pending
01:07:18,415 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 122.5 μs
01:07:18,415 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: waiting to receive commands
01:07:18,415 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower779: running 1 completion(s)
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): sendAppendEntries: 2 is present for follower follower780
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader778 (Leader): Sending AppendEntries to follower follower780: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=5, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}, JournaledLogEntry{index=3, term=6, command=three, pending=false}, JournaledLogEntry{index=4, term=6, command=four, pending=false}, JournaledLogEntry{index=5, term=6, command=five, pending=false}]}
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader778, prevLogIndex=1, prevLogTerm=6, leaderCommit=5, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}, JournaledLogEntry{index=3, term=6, command=three, pending=false}, JournaledLogEntry{index=4, term=6, command=four, pending=false}, JournaledLogEntry{index=5, term=6, command=five, pending=false}]}
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Number of entries to be appended = 4
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Append entry to log two
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=false} 
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Append entry to log three
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=false} 
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Append entry to log four
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=false} 
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Append entry to log five
01:07:18,489 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=false} 
01:07:18,489 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 2 commands after 423.3 ms
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Log size is now 5
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): Commit index set to 5
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower780, logLastIndex=5, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower780 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 1, lastIndex(): 5
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower780 (Follower): Setting last applied to 2
01:07:18,490 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower780 updated: matchIndex: 5, nextIndex: 6
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from 1 to 2
01:07:18,490 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader778: fakeSnapshot purging log to 4 for term 6
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower780: Applying state for log index 2 data two
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower780: applyState called: two
01:07:18,490 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader778 (Leader): Checking sendAppendEntries for follower follower780: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower780 (Follower): Setting last applied to 3
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from 2 to 3
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower780: Applying state for log index 3 data three
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower780: applyState called: three
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower780 (Follower): Setting last applied to 4
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from 3 to 4
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower780: Applying state for log index 4 data four
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower780: applyState called: four
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower780 (Follower): Setting last applied to 5
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower780: Moving last applied index from 4 to 5
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower780: Applying state for log index 5 data five
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower780: applyState called: five
01:07:18,490 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower780: fakeSnapshot purging log to 1 for term 6
01:07:18,494 PM [follower780-writer-166] [DEBUG] RaftStorageCompleter - follower780: 2 completion(s) pending
01:07:18,494 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 2 commands in 4.360 ms
01:07:18,494 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 3 commands after 5.560 μs
01:07:18,494 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower780: running 2 completion(s)
01:07:18,494 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false}
01:07:18,494 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false}
01:07:18,497 PM [follower780-writer-166] [DEBUG] RaftStorageCompleter - follower780: 3 completion(s) pending
01:07:18,497 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 3 commands in 2.937 ms
01:07:18,497 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: waiting to receive commands
01:07:18,497 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower780: running 3 completion(s)
01:07:18,497 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false}
01:07:18,497 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower780: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false}
01:07:18,609 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,609 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower780: fakeSnapshot purging log to 4 for term 6
01:07:18,609 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,609 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower779: fakeSnapshot purging log to 4 for term 6
01:07:18,730 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,730 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,849 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower780: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,849 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader778 and reached 5 (of expected -1)
01:07:18,865 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaAppendEntries complete
01:07:18,865 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower779-collector781#-1839486195]
01:07:18,876 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower779]
01:07:18,876 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower779: terminating thread VirtualThread[#945,follower779-writer-165]/waiting
01:07:18,877 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: received 1 commands after 461.6 ms
01:07:18,877 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: completed 1 commands in 18.09 μs
01:07:18,877 PM [follower779-writer-165] [DEBUG] JournalWriteTask - follower779: journal writer stopped
01:07:18,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower779: journal closed
01:07:18,877 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower779: journal closed
01:07:18,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks
01:07:18,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks
01:07:18,877 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower780-collector782#1495700860]
01:07:18,878 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower780]
01:07:18,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower780: terminating thread VirtualThread[#946,follower780-writer-166]/waiting
01:07:18,878 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: received 1 commands after 380.9 ms
01:07:18,878 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: completed 1 commands in 12.98 μs
01:07:18,878 PM [follower780-writer-166] [DEBUG] JournalWriteTask - follower780: journal writer stopped
01:07:18,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower780: journal closed
01:07:18,878 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower780: journal closed
01:07:18,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower780: stopped executor with 0 remaining tasks
01:07:18,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower780: stopped executor with 0 remaining tasks
01:07:18,878 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader778-collector783#1183828888]
01:07:18,879 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader778]
01:07:18,879 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - leader778: terminating thread VirtualThread[#947,leader778-writer-167]/waiting
01:07:18,879 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: received 1 commands after 464.4 ms
01:07:18,879 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: completed 1 commands in 12.92 μs
01:07:18,879 PM [leader778-writer-167] [DEBUG] JournalWriteTask - leader778: journal writer stopped
01:07:18,879 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader778: journal closed
01:07:18,879 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader778: journal closed
01:07:18,879 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader778: stopped executor with 0 remaining tasks
01:07:18,879 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader778: stopped executor with 0 remaining tasks
01:07:18,881 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderInstallsSnapshotWithRestartedFollowerDuringSnapshotInstallation starting
[WARN] [07/01/2025 13:07:18.882] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower788]: org.opendaylight.controller.cluster.common.actor.Monitor@26e55703
01:07:18,883 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower788: Starting recovery with journal batch size 1000
01:07:18,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: started executor
01:07:18,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: started executor
01:07:18,884 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower788: metadata journal open: replayFrom=1 applyTo=0
01:07:18,884 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower788: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:18,884 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower788: journal open: applyTo=0
01:07:18,885 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: journal writer started
01:07:18,885 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:18,886 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower788/TermInfo.properties does not exist
01:07:18,886 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower788/journal-v1-1.log
01:07:18,886 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower788/metadata-v1-1.log
01:07:18,886 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower788: no eligible files found
01:07:18,886 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower788: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
[WARN] [07/01/2025 13:07:18.886] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower789]: org.opendaylight.controller.cluster.common.actor.Monitor@620b5e27
01:07:18,886 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower789: Starting recovery with journal batch size 1000
01:07:18,887 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower789: started executor
01:07:18,887 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower789: started executor
01:07:18,887 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower789: metadata journal open: replayFrom=1 applyTo=0
01:07:18,887 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower789: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:18,887 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower789: journal open: applyTo=0
01:07:18,887 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: journal writer started
01:07:18,887 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: waiting to receive commands
01:07:18,888 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties does not exist
01:07:18,888 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:18,888 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:18,888 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:18,888 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower789: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
[WARN] [07/01/2025 13:07:18.888] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader787]: org.opendaylight.controller.cluster.common.actor.Monitor@62d094e8
01:07:18,889 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - leader787: Starting recovery with journal batch size 1000
01:07:18,889 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader787: started executor
01:07:18,889 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader787: started executor
01:07:18,889 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader787: metadata journal open: replayFrom=1 applyTo=0
01:07:18,890 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader787: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:18,890 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader787: journal open: applyTo=0
01:07:18,890 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: journal writer started
01:07:18,890 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:18,891 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/TermInfo.properties does not exist
01:07:18,891 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/journal-v1-1.log
01:07:18,891 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/metadata-v1-1.log
01:07:18,891 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader787: no eligible files found
01:07:18,891 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader787: Set currentTerm=5, votedFor=leader787
01:07:18,891 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader787: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:18,892 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=0, votedFor=null
01:07:18,892 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower788: Local TermInfo store seeded with TermInfo{term=0}
01:07:18,892 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower788: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1c0ed995
01:07:18,892 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to -1
01:07:18,892 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower788: journal recovery completedin 234.0 μs with journalIndex=1
01:07:18,895 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=0, votedFor=null
01:07:18,895 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower789: Local TermInfo store seeded with TermInfo{term=0}
01:07:18,895 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower789: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3cb53609
01:07:18,895 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower789: Moving last applied index from -1 to -1
01:07:18,895 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - follower789: journal recovery completedin 286.3 μs with journalIndex=1
01:07:18,895 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader787: Set currentTerm=5, votedFor=leader787
01:07:18,896 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader787: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader787}
01:07:18,896 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader787: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7f891bbf
01:07:18,896 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from -1 to -1
01:07:18,896 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - leader787: journal recovery completedin 154.5 μs with journalIndex=1
01:07:19,379 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
01:07:19,379 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader787 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.139:17355, Removed)
01:07:19,379 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader787 (Follower): Received ElectionTimeout - switching to Candidate
01:07:19,379 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader787 (Candidate): Election: Candidate has following voting peers: [follower788, follower789]
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader787: Set currentTerm=6, votedFor=leader787
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader787 (Candidate): Starting new election term 6
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader787 (Candidate): Sending RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1} to peer follower788
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader787 (Candidate): Sending RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1} to peer follower789
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader787 (Follower) :- Switching from behavior Follower to Candidate, election term: 6
01:07:19,383 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower788 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:19,383 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower788 (Follower): Cluster state: []
01:07:19,383 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower788 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:19,383 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower788 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower789 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower789 (Follower): Cluster state: []
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower789 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:19,383 PM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower789 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:19,386 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=6, votedFor=null
01:07:19,386 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=null
01:07:19,386 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower789 (Follower): In requestVote: RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:19,386 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower788 (Follower): In requestVote: RequestVote{term=6, candidateId=leader787, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=6, votedFor=leader787
01:07:19,388 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=leader787
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower789 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:19,388 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower788 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader787 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader787 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): Election: Leader has following peers: [follower788, follower789]
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader787 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6
01:07:19,388 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Last sync leader does not match current leader leader787, need to catch up to -1
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: -1, nextIndex: 0
01:07:19,388 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:19,389 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower789: Last sync leader does not match current leader leader787, need to catch up to -1
01:07:19,389 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower789 updated: matchIndex: -1, nextIndex: 0
01:07:19,389 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [07/01/2025 13:07:19.389] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=false]
[WARN] [07/01/2025 13:07:19.389] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower789] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower789, initialSyncDone=false]
01:07:19,401 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected
01:07:19,401 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader787: Persist data index=0 term=6 command=zero
01:07:19,401 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:19,401 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
01:07:19,401 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:19,401 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 511.5 ms
01:07:19,401 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 0 is present for follower follower788
01:07:19,401 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:19,401 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:19,401 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Append entry to log zero
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Log size is now 1
01:07:19,402 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 516.9 ms
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached -1 (of expected -1)
[WARN] [07/01/2025 13:07:19.402] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=true]
01:07:19,402 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:19,402 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 0 is present for follower follower789
01:07:19,402 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower789: AppendEntries{term=6, leaderId=leader787, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): Number of entries to be appended = 1
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): Append entry to log zero
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower789: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): Log size is now 1
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:19,402 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached -1 (of expected -1)
01:07:19,402 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: received 1 commands after 515.0 ms
[WARN] [07/01/2025 13:07:19.402] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower789] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower789, initialSyncDone=true]
01:07:19,403 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:19,407 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:19,407 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 5.706 ms
01:07:19,407 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true}
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@620484cf
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 0, nextIndex: 1
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 0
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from -1 to 0
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader787: Applying state for log index 0 data zero
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader787: applyState called: zero
01:07:19,407 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:19,408 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 505.7 μs
01:07:19,408 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower789 updated: matchIndex: 0, nextIndex: 1
01:07:19,408 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:19,408 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:19,408 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Commit index set to 0
01:07:19,408 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 140.4 μs
01:07:19,408 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader787: Persist data index=1 term=6 command=one
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): Commit index set to 0
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached 0 (of expected -1)
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:19,408 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 165.2 μs
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower789 (Follower): Setting last applied to 0
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 1 is present for follower follower788
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower789: Moving last applied index from -1 to 0
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower789: Applying state for log index 0 data zero
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower789: applyState called: zero
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 1 is present for follower follower789
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower789: AppendEntries{term=6, leaderId=leader787, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:19,408 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:19,408 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:19,408 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 6.522 ms
01:07:19,408 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:19,408 PM [follower789-writer-169] [DEBUG] RaftStorageCompleter - follower789: 1 completion(s) pending
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:19,408 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 5.846 ms
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): Number of entries to be appended = 1
01:07:19,408 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: received 1 commands after 3.660 μs
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): Append entry to log one
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower789: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): Log size is now 2
01:07:19,408 PM [follower789-writer-169] [DEBUG] RaftStorageCompleter - follower789: 1 more completion(s) pending
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower789 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached 0 (of expected -1)
01:07:19,408 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 109.1 μs
01:07:19,408 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: received 1 commands after 3.590 μs
01:07:19,408 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 0 (of expected -1)
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower789: running 2 completion(s)
01:07:19,408 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:19,408 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower789: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 0
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to 0
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower788: Applying state for log index 0 data zero
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower788: applyState called: zero
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:19,409 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 509.7 μs
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Append entry to log one
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Log size is now 2
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:19,409 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 0 (of expected -1)
01:07:19,409 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:19,409 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 322.1 μs
01:07:19,409 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 4.790 μs
01:07:19,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:19,410 PM [follower789-writer-169] [DEBUG] RaftStorageCompleter - follower789: 1 completion(s) pending
01:07:19,410 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 1.710 ms
01:07:19,410 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: waiting to receive commands
01:07:19,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower789: running 1 completion(s)
01:07:19,410 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:19,410 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower789: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:19,410 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:19,410 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 2.377 ms
01:07:19,410 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:19,410 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true}
01:07:19,410 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@4a161f8c
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 1 is present for follower follower789
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower789 updated: matchIndex: 1, nextIndex: 2
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 0 to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader787: Applying state for log index 1 data one
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader787: applyState called: one
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader787: fakeSnapshot purging log to 0 for term 6
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:19,411 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 359.9 μs
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 1 is present for follower follower788
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=0, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=false}]}
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): Commit index set to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
01:07:19,411 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower789 (Follower): Setting last applied to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower789: Moving last applied index from 0 to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower789: Applying state for log index 1 data one
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower789: applyState called: one
01:07:19,411 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 194.6 μs
01:07:19,411 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower789: fakeSnapshot purging log to 0 for term 6
01:07:19,411 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: received 1 commands after 880.5 μs
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=0, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=false}]}
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): matchEntry JournaledLogEntry{index=1, term=6, command=one, pending=false} is present: existingEntryTerm: 6
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 1, entries to be added from: 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Log size is now 2
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Commit index set to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:19,411 PM [follower789-writer-169] [DEBUG] RaftStorageCompleter - follower789: 1 completion(s) pending
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 1
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 0 to 1
01:07:19,411 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 105.2 μs
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower788: Applying state for log index 1 data one
01:07:19,411 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: waiting to receive commands
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower788: applyState called: one
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower788: fakeSnapshot purging log to 0 for term 6
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 1, nextIndex: 2
01:07:19,411 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:19,412 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower789: running 1 completion(s)
01:07:19,412 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:19,412 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 2.438 ms
01:07:19,412 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 3.461 μs
01:07:19,412 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 more completion(s) pending
01:07:19,412 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 72.71 μs
01:07:19,412 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:19,412 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - follower788: running 2 completion(s)
01:07:19,412 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:19,619 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
01:07:19,619 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
01:07:19,653 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - follower789: terminating thread VirtualThread[#951,follower789-writer-169]/waiting
01:07:19,653 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: received 1 commands after 242.0 ms
01:07:19,653 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 15.93 μs
01:07:19,653 PM [follower789-writer-169] [DEBUG] JournalWriteTask - follower789: journal writer stopped
01:07:19,654 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower789: journal closed
01:07:19,654 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower789: journal closed
01:07:19,654 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower789: stopped executor with 0 remaining tasks
01:07:19,654 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower789: stopped executor with 0 remaining tasks
01:07:19,739 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
[WARN] [07/01/2025 13:07:19.739] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:19,859 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
[WARN] [07/01/2025 13:07:19.859] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:19,979 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
[WARN] [07/01/2025 13:07:19.979] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,58 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader787: Persist data index=2 term=6 command=two
01:07:20,58 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 
01:07:20,58 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 647.5 ms
01:07:20,59 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
01:07:20,59 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
01:07:20,59 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 2 is present for follower follower788
01:07:20,59 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]}
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): Append entry to log two
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): Log size is now 2
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:20,59 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 1 (of expected -1)
01:07:20,59 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 647.7 ms
01:07:20,60 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
01:07:20,60 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
[WARN] [07/01/2025 13:07:20.060] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,62 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,62 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 3.367 ms
01:07:20,62 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true}
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@1c51ef09
01:07:20,62 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 2, nextIndex: 3
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 2
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 1 to 2
01:07:20,62 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 2.645 ms
01:07:20,62 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader787: Applying state for log index 2 data two
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader787: applyState called: two
01:07:20,62 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,62 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false}
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader787: fakeSnapshot purging log to 1 for term 6
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
01:07:20,62 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 459.0 μs
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader787: Persist data index=3 term=6 command=three
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 
01:07:20,62 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true
01:07:20,63 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:20,63 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 3 is present for follower follower788
01:07:20,63 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:20,63 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:20,63 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,63 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,63 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 199.8 μs
01:07:20,63 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:20,63 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 3.261 μs
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Commit index set to 2
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 2 (of expected -1)
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 2
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 1 to 2
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower788: Applying state for log index 2 data two
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower788: applyState called: two
[WARN] [07/01/2025 13:07:20.063] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower788: fakeSnapshot purging log to 1 for term 6
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]}
01:07:20,63 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 780.7 μs
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Append entry to log three
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Log size is now 2
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 2 (of expected -1)
01:07:20,63 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,63 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 171.1 μs
01:07:20,63 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,63 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 3.530 μs
01:07:20,65 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,65 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 2.106 ms
01:07:20,65 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,65 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,65 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,65 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 1.740 ms
01:07:20,65 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:20,65 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true}
01:07:20,65 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,65 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false}
01:07:20,65 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader787: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false]
01:07:20,65 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader787: captured snapshot at lastSequenceNumber: 4
01:07:20,65 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader787: Persising snapshot at EntryInfo[index=2, term=6]/EntryInfo[index=3, term=6]
01:07:20,65 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader787: support() called
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader787: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 6
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@485c17a0
01:07:20,66 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.tmp
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 3 is present for follower follower788
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 3, nextIndex: 4
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 3
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 2 to 3
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader787: Applying state for log index 3 data three
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader787: applyState called: three
01:07:20,66 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1
01:07:20,67 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 1.726 ms
01:07:20,67 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,67 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 176.9 μs
01:07:20,67 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Commit index set to 3
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 3 (of expected -1)
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 3
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 2 to 3
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower788: Applying state for log index 3 data three
01:07:20,67 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower788: applyState called: three
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader787: Persist data index=4 term=6 command=four
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true
01:07:20,67 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 367.6 μs
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 4 is present for follower follower788
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=3, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=6, command=four, pending=true}]}
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:20,67 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 2.474 ms
01:07:20,67 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:20,68 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,68 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 114.3 μs
01:07:20,68 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
[WARN] [07/01/2025 13:07:20.068] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=3, prevLogTerm=6, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=6, command=four, pending=true}]}
01:07:20,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/TermInfo.properties
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/journal-v1-1.log
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/metadata-v1-1.log
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorage - leader787: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:20.066158487Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1, size=510, entryCompress=NONE, stateCompress=NONE}
01:07:20,68 PM [leader787-%d365312] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,68 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 3, entries to be added from: 0
01:07:20,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Append entry to log four
01:07:20,68 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: snapshot is durable as of 2025-07-01T13:07:20.066158487Z
01:07:20,68 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 
01:07:20,68 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader787: Snapshot success -  sequence number: 4
01:07:20,68 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:20,68 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 877.6 μs
01:07:20,69 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): Log size is now 3
01:07:20,69 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=4, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:20,69 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 3 (of expected -1)
01:07:20,69 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=true}
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 1.629 ms
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 4.680 μs
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@51b20817
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorSnapshotMessageSupport - leader787: SnapshotComplete received
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 4, nextIndex: 5
01:07:20,69 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 4
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 111.0 μs
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 3 to 4
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader787: Applying state for log index 4 data four
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader787: applyState called: four
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 1
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 188.6 μs
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader787: Persist data index=5 term=6 command=five
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Commit index set to 4
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 4 (of expected -1)
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 5 is present for follower follower788
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 4
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=4, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:20,69 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 3 to 4
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower788: Applying state for log index 4 data four
01:07:20,69 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower788: applyState called: four
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 199.8 μs
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,69 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:20,69 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 15.86 μs
[WARN] [07/01/2025 13:07:20.069] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=4, prevLogTerm=6, leaderCommit=4, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=5, term=6, command=five, pending=true}]}
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 4, entries to be added from: 0
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Append entry to log five
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Log size is now 4
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=5, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:20,70 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 4 (of expected -1)
01:07:20,70 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,70 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 1.667 ms
01:07:20,70 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 2 commands after 3.440 μs
01:07:20,70 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,70 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,70 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 735.4 μs
01:07:20,70 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=true}
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@5180daa3
01:07:20,70 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false}
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 5 is present for follower follower788
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 5, nextIndex: 6
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 5
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 4 to 5
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader787: Applying state for log index 5 data five
01:07:20,70 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader787: applyState called: five
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 1
01:07:20,71 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 305.0 μs
01:07:20,71 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): Commit index set to 5
01:07:20,71 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 59.96 μs
01:07:20,71 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 5 (of expected -1)
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 5
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 4 to 5
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower788: Applying state for log index 5 data five
01:07:20,71 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - follower788: applyState called: five
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader787: Persist data index=6 term=6 command=six
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true
01:07:20,71 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 313.3 μs
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 1
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 6 is present for follower follower788
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=5, prevLogTerm=6, leaderCommit=5, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]}
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 1
01:07:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=5, prevLogTerm=6, leaderCommit=5, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]}
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
[WARN] [07/01/2025 13:07:20.071] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 5, entries to be added from: 0
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Append entry to log six
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Log size is now 5
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:20,71 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 5 (of expected -1)
01:07:20,72 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 2 completion(s) pending
01:07:20,72 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - follower788: running 2 completion(s)
01:07:20,72 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 2 commands in 1.695 ms
01:07:20,72 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false}
01:07:20,72 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 2 commands after 3.430 μs
01:07:20,73 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 2 completion(s) pending
01:07:20,73 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - follower788: running 2 completion(s)
01:07:20,73 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 2 commands in 727.5 μs
01:07:20,73 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:20,73 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false}
01:07:20,73 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,73 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 1.809 ms
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,73 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=true}
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@49a2a32c
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 6, nextIndex: 7
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 6
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 5 to 6
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader787: Applying state for log index 6 data six
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader787: applyState called: six
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1
01:07:20,73 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 304.1 μs
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Commit index set to 6
01:07:20,73 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 6 (of expected -1)
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6
01:07:20,73 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 103.3 μs
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 6
01:07:20,73 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 5 to 6
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower788: Applying state for log index 6 data six
01:07:20,73 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower788: applyState called: six
01:07:20,74 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 1.278 ms
01:07:20,74 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,74 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 171.3 μs
01:07:20,74 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:20,74 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,73 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
[WARN] [07/01/2025 13:07:20.099] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,159 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1
01:07:20,159 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/TermInfo.properties
01:07:20,159 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/journal-v1-1.log
01:07:20,159 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/metadata-v1-1.log
01:07:20,159 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: picked SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:20.066158487Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1, size=510, entryCompress=NONE, stateCompress=NONE} as the latest file
01:07:20,160 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderInstallsSnapshotWithRestartedFollowerDuringSnapshotInstallation: sending 1 more payload to trigger second snapshot
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader787: Persist data index=7 term=6 command=seven
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader787: Append log entry and persist JournaledLogEntry{index=7, term=6, command=seven, pending=true} 
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Replicate message: identifier: MockIdentifier{value=seven}, logIndex: 7, isSendImmediate: true
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 1
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): sendAppendEntries: 7 is present for follower follower788
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader787, prevLogIndex=6, prevLogTerm=6, leaderCommit=6, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=6, command=seven, pending=true}]}
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower789: active: false, followerNextIndex: 2, leaderLastIndex: 7, leaderSnapShotIndex: 1
01:07:20,160 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: awaiting more completions to resolve 1 synchronized callback(s)
01:07:20,160 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader787, prevLogIndex=6, prevLogTerm=6, leaderCommit=6, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=6, command=seven, pending=true}]}
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0
01:07:20,161 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 87.23 ms
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Append entry to log seven
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=7, term=6, command=seven, pending=true} 
[WARN] [07/01/2025 13:07:20.161] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): Log size is now 6
01:07:20,161 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 86.52 ms
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=7, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:20,161 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 6 (of expected -1)
01:07:20,162 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,162 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,162 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 1.589 ms
01:07:20,162 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:20,162 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 1.741 ms
01:07:20,162 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,162 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,162 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=7, term=6, command=seven, pending=false}
01:07:20,162 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,162 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - leader787: persist complete JournaledLogEntry{index=7, term=6, command=seven, pending=true}
01:07:20,162 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=6, lastAppliedTerm=6, lastIndex=7, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false]
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader787: captured snapshot at lastSequenceNumber: 8
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: Persising snapshot at EntryInfo[index=6, term=6]/EntryInfo[index=7, term=6]
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader787: support() called
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader787: log size 6 exceeds the snapshot batch count 4 - doing snapshotPreCommit with index 6
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 6 and term: 6
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@168e4190
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 7, nextIndex: 8
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - leader787 (Leader): Setting last applied to 7
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader787: Moving last applied index from 6 to 7
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader787: Applying state for log index 7 data seven
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader787: applyState called: seven
01:07:20,163 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Commit index set to 7
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 7
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 6 to 7
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower788: Applying state for log index 7 data seven
01:07:20,164 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.tmp
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower788: applyState called: seven
01:07:20,164 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 1.417 ms
01:07:20,164 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 1.539 ms
01:07:20,164 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,164 PM [follower788-writer-168] [DEBUG] RaftStorageCompleter - follower788: 1 completion(s) pending
01:07:20,164 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 77.61 μs
01:07:20,164 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: waiting to receive commands
01:07:20,164 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 151.6 μs
01:07:20,164 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,164 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower788: running 1 completion(s)
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/TermInfo.properties
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/journal-v1-1.log
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/metadata-v1-1.log
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: deleted snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=6], timestamp=2025-07-01T13:07:20.066158487Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-03F17F97.v1, size=510, entryCompress=NONE, stateCompress=NONE}
01:07:20,166 PM [leader787-%d365313] [DEBUG] RaftStorage - leader787: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=6, term=6], timestamp=2025-07-01T13:07:20.163500995Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1, size=578, entryCompress=NONE, stateCompress=NONE}
01:07:20,167 PM [leader787-%d365313] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,167 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,167 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader787: snapshot is durable as of 2025-07-01T13:07:20.163500995Z
01:07:20,167 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader787: Snapshot success -  sequence number: 8
01:07:20,167 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorSnapshotMessageSupport - leader787: SnapshotComplete received
01:07:20,167 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 2.743 ms
01:07:20,167 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,167 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 189.8 μs
01:07:20,167 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,167 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
[WARN] [07/01/2025 13:07:20.219] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
[WARN] [07/01/2025 13:07:20.340] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower789] received dead letter from TestActor[pekko://test/user/leader787]: AppendEntries{term=6, leaderId=leader787, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}
01:07:20,340 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,362 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1
01:07:20,362 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/TermInfo.properties
01:07:20,362 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/journal-v1-1.log
01:07:20,362 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/metadata-v1-1.log
01:07:20,362 PM [awaitility-thread] [DEBUG] RaftStorage - leader787: picked SnapshotFileV1{lastIncluded=EntryInfo[index=6, term=6], timestamp=2025-07-01T13:07:20.163500995Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1, size=578, entryCompress=NONE, stateCompress=NONE} as the latest file
[WARN] [07/01/2025 13:07:20.365] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower789]: org.opendaylight.controller.cluster.common.actor.Monitor@5be69005
01:07:20,366 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower789: Starting recovery with journal batch size 1000
01:07:20,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: started executor
01:07:20,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: started executor
01:07:20,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower789: metadata journal open: replayFrom=1 applyTo=2
01:07:20,368 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower789: journal open: firstIndex=1 lastIndex=2 replayFrom=1 fileEntries=0
01:07:20,368 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower789: journal open: applyTo=2
01:07:20,368 PM [follower789-writer-171] [DEBUG] JournalWriteTask - follower789: journal writer started
01:07:20,368 PM [follower789-writer-171] [DEBUG] JournalWriteTask - follower789: waiting to receive commands
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=6, votedFor=leader787
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower789: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRecovery - follower789: restoring local TermInfo{term=6, votedFor=leader787}
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=6, votedFor=leader787
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower789: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@29b4cca0
01:07:20,369 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower789: Moving last applied index from -1 to -1
01:07:20,370 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower789: recovered journal DefaultLogEntry[index=0, term=6, command=zero]
01:07:20,370 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower789: recovered journal DefaultLogEntry[index=1, term=6, command=one]
01:07:20,370 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower789: journal recovery completedin 967.6 μs with journalIndex=1
01:07:20,460 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,460 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower789: Last sync leader does not match current leader leader787, need to catch up to -1
01:07:20,460 PM [test-pekko.actor.default-dispatcher-7] [WARN] AbstractLeader - leader787 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 840, lastApplied : 7, commitIndex : 7
01:07:20,460 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractLeader - leader787 (Leader): Initiating install snapshot to follower follower789: follower nextIndex: 2, leader snapshotIndex: 6, leader lastIndex: 7, leader log size: 1
[WARN] [07/01/2025 13:07:20.460] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower789] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower789, initialSyncDone=false]
01:07:20,460 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=7, lastAppliedTerm=6, lastIndex=7, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower789
01:07:20,460 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader787: lastSequenceNumber prior to capture: 9
01:07:20,460 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader787: support() called
01:07:20,461 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached -1 (of expected -1)
[WARN] [07/01/2025 13:07:20.461] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower789] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower789, initialSyncDone=true]
01:07:20,461 PM [leader787-%d365314] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,461 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,461 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: Persising snapshot at EntryInfo[index=7, term=6]/EntryInfo[index=7, term=6]
01:07:20,461 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader787: support() called
01:07:20,461 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader787: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 6 and term: 6
01:07:20,461 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): sendInstallSnapshot
01:07:20,461 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-1B870799.tmp
01:07:20,462 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] LeaderInstallSnapshotState - leader787 (Leader): Snapshot 396 bytes, total chunks to send: 1
01:07:20,462 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] LeaderInstallSnapshotState - leader787 (Leader): Next chunk: total length=396, offset=0, size=396, hashCode=-153134944
01:07:20,462 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): next snapshot chunk size for follower follower789: 396
01:07:20,462 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader787 (Leader): InstallSnapshot sent to follower Vector(user, follower789), Chunk: 1/1
01:07:20,462 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower789 (Follower): handleInstallSnapshot: InstallSnapshot{term=6, leaderId=leader787, lastIncludedIndex=7, lastIncludedTerm=6, datasize=396, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}
01:07:20,462 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower789: Lagging 8 entries behind leader leader787 and reached -1 (of expected -1)
01:07:20,462 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotTracker - follower789 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
01:07:20,462 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower789 (Follower): Snapshot received from leader: leader787
01:07:20,462 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - follower789: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=396}}
01:07:20,462 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower789: support() called
01:07:20,463 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower789: Converted InstallSnapshot from leader: leader787 to state
01:07:20,463 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower789: lastSequenceNumber prior to persisting applied snapshot: 3
01:07:20,463 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower789: support() called
01:07:20,463 PM [follower789-%d292383] [DEBUG] RaftStorage - follower789: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD88-1BA2C649.tmp
01:07:20,465 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-1B870799.v1
01:07:20,465 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/TermInfo.properties
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/journal-v1-1.log
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/metadata-v1-1.log
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-1B870799.v1
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: deleted snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=6, term=6], timestamp=2025-07-01T13:07:20.163500995Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-09BED3C3.v1, size=578, entryCompress=NONE, stateCompress=NONE}
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorage - leader787: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=7, term=6], timestamp=2025-07-01T13:07:20.461834137Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/leader787/snapshot-000000006863DD88-1B870799.v1, size=460, entryCompress=NONE, stateCompress=NONE}
01:07:20,466 PM [leader787-%d365315] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,466 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,466 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PekkoRaftStorage - follower789: terminating thread VirtualThread[#958,follower789-writer-171]/waiting
01:07:20,466 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader787: snapshot is durable as of 2025-07-01T13:07:20.461834137Z
01:07:20,466 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader787: Snapshot success -  sequence number: 9
01:07:20,466 PM [follower789-writer-171] [DEBUG] JournalWriteTask - follower789: received 1 commands after 98.58 ms
01:07:20,466 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorSnapshotMessageSupport - leader787: SnapshotComplete received
01:07:20,466 PM [follower789-writer-171] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 12.55 μs
01:07:20,466 PM [follower789-writer-171] [DEBUG] JournalWriteTask - follower789: journal writer stopped
01:07:20,466 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 299.4 ms
01:07:20,467 PM [leader787-writer-170] [DEBUG] RaftStorageCompleter - leader787: 1 completion(s) pending
01:07:20,467 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 172.7 μs
01:07:20,467 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: waiting to receive commands
01:07:20,467 PM [follower789-%d292383] [DEBUG] RaftStorageCompleter - follower789: 1 completion(s) pending
01:07:20,467 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorageCompleter - leader787: running 1 completion(s)
01:07:20,467 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower789: journal closed
01:07:20,467 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower789: journal closed
01:07:20,467 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower789: stopped executor with 0 remaining tasks
01:07:20,467 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower789: stopped executor with 0 remaining tasks
[WARN] [07/01/2025 13:07:20.468] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower789] received dead letter: ExecuteInSelfMessage[runnable=org.opendaylight.controller.cluster.raft.spi.RaftStorageCompleter$$Lambda/0x00000007c0806f88@48d1d5a5]
[WARN] [07/01/2025 13:07:20.569] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower789]: org.opendaylight.controller.cluster.common.actor.Monitor@43397f58
01:07:20,570 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - follower789: Starting recovery with journal batch size 1000
01:07:20,570 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: started executor
01:07:20,570 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower789: started executor
01:07:20,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower789: metadata journal open: replayFrom=1 applyTo=2
01:07:20,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower789: journal open: firstIndex=1 lastIndex=2 replayFrom=1 fileEntries=0
01:07:20,572 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower789: journal open: applyTo=2
01:07:20,572 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: journal writer started
01:07:20,572 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: waiting to receive commands
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=6, votedFor=leader787
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower789: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRecovery - follower789: restoring local TermInfo{term=6, votedFor=leader787}
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower789: Set currentTerm=6, votedFor=leader787
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower789: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@270154f8
01:07:20,573 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower789: Moving last applied index from -1 to -1
01:07:20,574 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - follower789: recovered journal DefaultLogEntry[index=0, term=6, command=zero]
01:07:20,575 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - follower789: recovered journal DefaultLogEntry[index=1, term=6, command=one]
01:07:20,575 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - follower789: journal recovery completedin 1.300 ms with journalIndex=1
01:07:20,579 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,699 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,770 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:20,770 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:20,770 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:20,770 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:20,819 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,871 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:20,871 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:20,871 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:20,871 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:20,939 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:20,939 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower789: Last sync leader does not match current leader leader787, need to catch up to -1
01:07:20,939 PM [test-pekko.actor.default-dispatcher-18] [WARN] AbstractLeader - leader787 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 478, lastApplied : 7, commitIndex : 7
[WARN] [07/01/2025 13:07:20.940] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower789] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower789, initialSyncDone=false]
01:07:20,971 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:20,971 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:20,971 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:20,971 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,59 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,72 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,72 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,72 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,72 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,172 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,172 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,172 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,172 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,179 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,273 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,273 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,273 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,273 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,300 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,374 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,374 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,374 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,374 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,419 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,419 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached -1 (of expected -1)
01:07:21,419 PM [test-pekko.actor.default-dispatcher-6] [WARN] AbstractLeader - leader787 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 479, lastApplied : 7, commitIndex : 7
[WARN] [07/01/2025 13:07:21.419] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower789] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower789, initialSyncDone=true]
01:07:21,474 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,474 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,474 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,474 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,539 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,575 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,575 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,575 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,575 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,659 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,675 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,675 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,675 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,676 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,776 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,776 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,776 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,776 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,779 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:21,899 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached -1 (of expected -1)
01:07:21,899 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:21,899 PM [test-pekko.actor.default-dispatcher-12] [WARN] AbstractLeader - leader787 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 480, lastApplied : 7, commitIndex : 7
01:07:21,977 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:21,977 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:21,977 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:21,977 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:22,19 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:22,139 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:22,178 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,178 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,178 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,178 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:22,260 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:22,379 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,379 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,379 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:22,379 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,379 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower789: Lagging 0 entries behind leader leader787 and reached -1 (of expected -1)
01:07:22,379 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:22,380 PM [test-pekko.actor.default-dispatcher-7] [WARN] AbstractLeader - leader787 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=6, success=true, followerId=follower789, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 480, lastApplied : 7, commitIndex : 7
01:07:22,480 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,480 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,480 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,480 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: no eligible files found
01:07:22,499 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): next snapshot chunk size for follower follower789: 396
01:07:22,499 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader787 (Leader): InstallSnapshot sent to follower Vector(user, follower789), Chunk: 1/1
01:07:22,499 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader787 and reached 7 (of expected -1)
01:07:22,499 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower789 (Follower): handleInstallSnapshot: InstallSnapshot{term=6, leaderId=leader787, lastIncludedIndex=7, lastIncludedTerm=6, datasize=396, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}
01:07:22,499 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower789: Lagging 8 entries behind leader leader787 and reached -1 (of expected -1)
01:07:22,499 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotTracker - follower789 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
01:07:22,499 PM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower789 (Follower): Snapshot received from leader: leader787
01:07:22,500 PM [test-pekko.actor.default-dispatcher-12] [INFO] SnapshotManager - follower789: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=396}}
01:07:22,500 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower789: support() called
01:07:22,501 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower789: Converted InstallSnapshot from leader: leader787 to state
01:07:22,501 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower789: lastSequenceNumber prior to persisting applied snapshot: 3
01:07:22,501 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower789: support() called
01:07:22,501 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD8A-1DE257F3.tmp
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD8A-1DE257F3.v1
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD8A-1DE257F3.v1
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorage - follower789: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=7, term=6], timestamp=2025-07-01T13:07:22.501372915Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD8A-1DE257F3.v1, size=460, entryCompress=NONE, stateCompress=NONE}
01:07:22,505 PM [follower789-%d728868] [DEBUG] RaftStorageCompleter - follower789: 1 completion(s) pending
01:07:22,505 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorageCompleter - follower789: running 1 completion(s)
01:07:22,506 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - follower789: snapshot is durable as of 2025-07-01T13:07:22.501372915Z
01:07:22,506 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower789: Snapshot success -  sequence number: 3
01:07:22,506 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower789: support() called
01:07:22,506 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower789: applySnapshot called
01:07:22,506 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower789 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=6, followerId=follower789, chunkIndex=1, success=true}
01:07:22,506 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorSnapshotMessageSupport - follower789: SnapshotComplete received
01:07:22,506 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader787 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=6, followerId=follower789, chunkIndex=1, success=true}
01:07:22,506 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractLeader - leader787 (Leader): Snapshot successfully installed on follower follower789 (last chunk 1) - matchIndex set to 7, nextIndex set to 8
01:07:22,507 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: received 1 commands after 1.935 s
01:07:22,507 PM [follower789-writer-172] [DEBUG] RaftStorageCompleter - follower789: 1 completion(s) pending
01:07:22,507 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 324.9 μs
01:07:22,507 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: waiting to receive commands
01:07:22,507 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower789: running 1 completion(s)
01:07:22,581 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD8A-1DE257F3.v1
01:07:22,581 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/TermInfo.properties
01:07:22,581 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/journal-v1-1.log
01:07:22,581 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/metadata-v1-1.log
01:07:22,581 PM [awaitility-thread] [DEBUG] RaftStorage - follower789: picked SnapshotFileV1{lastIncluded=EntryInfo[index=7, term=6], timestamp=2025-07-01T13:07:22.501372915Z, file=/tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest2443265410414740778/follower789/snapshot-000000006863DD8A-1DE257F3.v1, size=460, entryCompress=NONE, stateCompress=NONE} as the latest file
01:07:22,581 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower788-collector790#344454606]
01:07:22,592 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower788]
01:07:22,593 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower788: terminating thread VirtualThread[#950,follower788-writer-168]/waiting
01:07:22,593 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: received 1 commands after 2.429 s
01:07:22,593 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: completed 1 commands in 14.03 μs
01:07:22,593 PM [follower788-writer-168] [DEBUG] JournalWriteTask - follower788: journal writer stopped
01:07:22,593 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower788: journal closed
01:07:22,593 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower788: journal closed
01:07:22,593 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks
01:07:22,593 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks
01:07:22,594 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower789-collector791#-2061359682]
01:07:22,594 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower789]
01:07:22,594 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader787-collector792#-1105794980]
01:07:22,594 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader787]
01:07:22,595 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - leader787: terminating thread VirtualThread[#952,leader787-writer-170]/waiting
01:07:22,595 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: received 1 commands after 2.128 s
01:07:22,595 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: completed 1 commands in 13.15 μs
01:07:22,595 PM [leader787-writer-170] [DEBUG] JournalWriteTask - leader787: journal writer stopped
01:07:22,595 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader787: journal closed
01:07:22,595 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader787: journal closed
01:07:22,595 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader787: stopped executor with 0 remaining tasks
01:07:22,595 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader787: stopped executor with 0 remaining tasks
01:07:22,595 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower789-collector793#-1155174494]
01:07:22,596 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower789]
01:07:22,596 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower789-collector794#-2041998666]
01:07:22,596 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower789]
01:07:22,596 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower789: terminating thread VirtualThread[#962,follower789-writer-172]/waiting
01:07:22,596 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: received 1 commands after 89.25 ms
01:07:22,596 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: completed 1 commands in 15.20 μs
01:07:22,596 PM [follower789-writer-172] [DEBUG] JournalWriteTask - follower789: journal writer stopped
01:07:22,596 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower789: journal closed
01:07:22,596 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower789: journal closed
01:07:22,596 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower789: stopped executor with 0 remaining tasks
01:07:22,596 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower789: stopped executor with 0 remaining tasks
01:07:22,598 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotTriggeredByMemoryThresholdExceededWithLaggingFollower starting
[WARN] [07/01/2025 13:07:22.600] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower799]: org.opendaylight.controller.cluster.common.actor.Monitor@3bffaff2
01:07:22,601 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower799: Starting recovery with journal batch size 1000
01:07:22,601 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower799: started executor
01:07:22,601 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower799: started executor
01:07:22,601 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower799: metadata journal open: replayFrom=1 applyTo=0
01:07:22,602 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower799: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:22,602 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower799: journal open: applyTo=0
01:07:22,602 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: journal writer started
01:07:22,602 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: waiting to receive commands
01:07:22,604 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/follower799/TermInfo.properties does not exist
[WARN] [07/01/2025 13:07:22.604] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower800]: org.opendaylight.controller.cluster.common.actor.Monitor@6d4bb9d5
01:07:22,604 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower799: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/follower799/journal-v1-1.log
01:07:22,604 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower799: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/follower799/metadata-v1-1.log
01:07:22,604 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower799: no eligible files found
01:07:22,604 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower799: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:22,605 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower800: Starting recovery with journal batch size 1000
01:07:22,605 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: started executor
01:07:22,605 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: started executor
01:07:22,605 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower800: metadata journal open: replayFrom=1 applyTo=0
01:07:22,606 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower800: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:22,606 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower800: journal open: applyTo=0
01:07:22,606 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: journal writer started
01:07:22,606 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: waiting to receive commands
01:07:22,606 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/follower800/TermInfo.properties does not exist
01:07:22,607 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/follower800/journal-v1-1.log
01:07:22,607 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/follower800/metadata-v1-1.log
01:07:22,607 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: no eligible files found
01:07:22,607 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower800: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:22,607 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower799: Set currentTerm=0, votedFor=null
01:07:22,607 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower799: Local TermInfo store seeded with TermInfo{term=0}
01:07:22,607 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower799: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@674028ac
01:07:22,607 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower799: Moving last applied index from -1 to -1
01:07:22,608 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower799: journal recovery completedin 361.7 μs with journalIndex=1
01:07:22,611 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower800: Set currentTerm=0, votedFor=null
01:07:22,611 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower800: Local TermInfo store seeded with TermInfo{term=0}
01:07:22,611 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower800: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@4b845aad
01:07:22,611 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - follower800: Moving last applied index from -1 to -1
01:07:22,611 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] JournalRecovery - follower800: journal recovery completedin 287.9 μs with journalIndex=1
[WARN] [07/01/2025 13:07:22.705] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader798]: org.opendaylight.controller.cluster.common.actor.Monitor@38812361
01:07:22,706 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader798: Starting recovery with journal batch size 1000
01:07:22,706 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader798: started executor
01:07:22,706 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader798: started executor
01:07:22,707 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader798: metadata journal open: replayFrom=1 applyTo=0
01:07:22,707 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader798: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
01:07:22,707 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader798: journal open: applyTo=0
01:07:22,708 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: journal writer started
01:07:22,708 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: waiting to receive commands
01:07:22,709 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/leader798/TermInfo.properties does not exist
01:07:22,710 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader798: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/leader798/journal-v1-1.log
01:07:22,710 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader798: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7603392334462919044/leader798/metadata-v1-1.log
01:07:22,710 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader798: no eligible files found
01:07:22,710 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader798: Set currentTerm=5, votedFor=leader798
01:07:22,710 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader798: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
01:07:22,714 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader798: Set currentTerm=5, votedFor=leader798
01:07:22,714 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - leader798: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader798}
01:07:22,714 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - leader798: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@e438624
01:07:22,714 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - leader798: Moving last applied index from -1 to -1
01:07:22,715 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] JournalRecovery - leader798: journal recovery completedin 353.4 μs with journalIndex=1
01:07:23,219 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
01:07:23,219 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - leader798 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.139:17355, Removed)
01:07:23,219 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - leader798 (Follower): Received ElectionTimeout - switching to Candidate
01:07:23,219 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader798 (Candidate): Election: Candidate has following voting peers: [follower799, follower800]
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - leader798: Set currentTerm=6, votedFor=leader798
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [INFO] Candidate - leader798 (Candidate): Starting new election term 6
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader798 (Candidate): Sending RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1} to peer follower799
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader798 (Candidate): Sending RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1} to peer follower800
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader798 (Follower) :- Switching from behavior Follower to Candidate, election term: 6
01:07:23,223 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower799 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower800 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
01:07:23,223 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower799 (Follower): Cluster state: []
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower800 (Follower): Cluster state: []
01:07:23,223 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower799 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower800 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
01:07:23,223 PM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower799 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:23,223 PM [test-pekko.actor.default-dispatcher-7] [INFO] Follower - follower800 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
01:07:23,229 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower800: Set currentTerm=6, votedFor=null
01:07:23,229 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower800 (Follower): In requestVote: RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:23,230 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower799: Set currentTerm=6, votedFor=null
01:07:23,230 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower799 (Follower): In requestVote: RequestVote{term=6, candidateId=leader798, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1
01:07:23,231 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower800: Set currentTerm=6, votedFor=leader798
01:07:23,231 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower800 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
01:07:23,231 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader798 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6
01:07:23,231 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Candidate - leader798 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1
01:07:23,231 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader798 (Leader): Election: Leader has following peers: [follower799, follower800]
01:07:23,232 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower799: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:23,232 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower800: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:23,232 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActorBehavior - leader798 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6
01:07:23,232 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower800: Last sync leader does not match current leader leader798, need to catch up to -1
01:07:23,232 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader798 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower800 updated: matchIndex: -1, nextIndex: 0
01:07:23,232 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower800: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
01:07:23,232 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower799: Set currentTerm=6, votedFor=leader798
01:07:23,232 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower799 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true}
[WARN] [07/01/2025 13:07:23.232] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower800] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower800, initialSyncDone=false]
01:07:23,232 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower799: Last sync leader does not match current leader leader798, need to catch up to -1
01:07:23,232 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower799 updated: matchIndex: -1, nextIndex: 0
01:07:23,232 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower799: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [07/01/2025 13:07:23.233] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower799] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower799, initialSyncDone=false]
01:07:23,269 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected
01:07:23,269 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader798: Persist data index=0 term=6 command=zero
01:07:23,269 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader798: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:23,269 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower799: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): sendAppendEntries: 0 is present for follower follower799
01:07:23,270 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: received 1 commands after 562.0 ms
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Sending AppendEntries to follower follower799: AppendEntries{term=6, leaderId=leader798, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower800: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): sendAppendEntries: 0 is present for follower follower800
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Sending AppendEntries to follower follower800: AppendEntries{term=6, leaderId=leader798, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:23,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader798: awaiting more completions to resolve 1 synchronized callback(s)
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader798, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): Number of entries to be appended = 1
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): Append entry to log zero
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower799: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): Log size is now 1
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower799, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:23,270 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached -1 (of expected -1)
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower800 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader798, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]}
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower800 (Follower): Number of entries to be appended = 1
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower800 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower800 (Follower): Append entry to log zero
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower800: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower800 (Follower): Log size is now 1
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower800 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower800, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:23,270 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower800: Lagging 0 entries behind leader leader798 and reached -1 (of expected -1)
[WARN] [07/01/2025 13:07:23.270] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower799] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower799, initialSyncDone=true]
01:07:23,270 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: received 1 commands after 668.0 ms
01:07:23,270 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: received 1 commands after 664.7 ms
[WARN] [07/01/2025 13:07:23.270] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower800] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower800, initialSyncDone=true]
01:07:23,277 PM [leader798-writer-175] [DEBUG] RaftStorageCompleter - leader798: 1 completion(s) pending
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader798: running 1 completion(s)
01:07:23,277 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: completed 1 commands in 7.000 ms
01:07:23,277 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: waiting to receive commands
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader798: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true}
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader798: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@6ad4eacd
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower799 updated: matchIndex: 0, nextIndex: 1
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader798 (Leader): Setting last applied to 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader798: Moving last applied index from -1 to 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader798: Applying state for log index 0 data zero
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader798: applyState called: zero
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower799: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:23,277 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: received 1 commands after 462.5 μs
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower800 updated: matchIndex: 0, nextIndex: 1
01:07:23,277 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower800: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
01:07:23,277 PM [leader798-writer-175] [DEBUG] RaftStorageCompleter - leader798: 1 completion(s) pending
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower800 (Follower): Commit index set to 0
01:07:23,277 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: completed 1 commands in 236.3 μs
01:07:23,277 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower799 (Follower): Commit index set to 0
01:07:23,277 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: waiting to receive commands
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower800: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower800 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower800 (Follower): Setting last applied to 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower800: Moving last applied index from -1 to 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower799 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower800: Applying state for log index 0 data zero
01:07:23,277 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower800: applyState called: zero
01:07:23,277 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower799 (Follower): Setting last applied to 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower799: Moving last applied index from -1 to 0
01:07:23,277 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower799: Applying state for log index 0 data zero
01:07:23,278 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower799: applyState called: zero
01:07:23,278 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - leader798: running 1 completion(s)
01:07:23,278 PM [follower799-writer-173] [DEBUG] RaftStorageCompleter - follower799: 1 completion(s) pending
01:07:23,278 PM [follower800-writer-174] [DEBUG] RaftStorageCompleter - follower800: 1 completion(s) pending
01:07:23,278 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: completed 1 commands in 7.436 ms
01:07:23,278 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: completed 1 commands in 7.305 ms
01:07:23,278 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: received 1 commands after 5.501 μs
01:07:23,278 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: received 1 commands after 31.16 μs
01:07:23,278 PM [follower799-writer-173] [DEBUG] RaftStorageCompleter - follower799: 1 more completion(s) pending
01:07:23,278 PM [follower800-writer-174] [DEBUG] RaftStorageCompleter - follower800: 1 more completion(s) pending
01:07:23,278 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: completed 1 commands in 150.6 μs
01:07:23,278 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: waiting to receive commands
01:07:23,278 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: completed 1 commands in 128.9 μs
01:07:23,278 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: waiting to receive commands
01:07:23,278 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - follower800: running 2 completion(s)
01:07:23,278 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower800: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:23,278 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorageCompleter - follower799: running 2 completion(s)
01:07:23,278 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower799: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false}
01:07:23,469 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower800: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,469 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,589 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,709 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,829 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader798: Persist data index=1 term=6 command=one
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader798: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower799: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): sendAppendEntries: 1 is present for follower follower799
01:07:23,877 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: received 1 commands after 599.5 ms
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): Sending AppendEntries to follower follower799: AppendEntries{term=6, leaderId=leader798, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower800: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
01:07:23,877 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader798: awaiting more completions to resolve 1 synchronized callback(s)
01:07:23,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower799 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader798, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]}
01:07:23,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower799 (Follower): Number of entries to be appended = 1
01:07:23,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower799 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
01:07:23,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower799 (Follower): Append entry to log one
01:07:23,877 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower799: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 
01:07:23,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower799 (Follower): Log size is now 2
01:07:23,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower799 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower799, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
01:07:23,878 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 0 (of expected -1)
01:07:23,878 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: received 1 commands after 599.6 ms
01:07:23,880 PM [leader798-writer-175] [DEBUG] RaftStorageCompleter - leader798: 1 completion(s) pending
01:07:23,880 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: completed 1 commands in 2.930 ms
01:07:23,880 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: waiting to receive commands
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader798: running 1 completion(s)
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader798: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true}
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorageCompleter - leader798: completed synchronized callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0810af0@7a7e68c4
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower799 updated: matchIndex: 1, nextIndex: 2
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader798 (Leader): Setting last applied to 1
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader798: Moving last applied index from 0 to 1
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader798: Applying state for log index 1 data one
01:07:23,880 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader798: applyState called: one
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader798: fakeSnapshot purging log to 0 for term 6
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader798 (Leader): Checking sendAppendEntries for follower follower799: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
01:07:23,881 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: received 1 commands after 605.1 μs
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): Commit index set to 1
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower799 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower799 (Follower): Setting last applied to 1
01:07:23,881 PM [leader798-writer-175] [DEBUG] RaftStorageCompleter - leader798: 1 completion(s) pending
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower799: Moving last applied index from 0 to 1
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower799: Applying state for log index 1 data one
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower799: applyState called: one
01:07:23,881 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: completed 1 commands in 258.2 μs
01:07:23,881 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: waiting to receive commands
01:07:23,881 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorageCompleter - leader798: running 1 completion(s)
01:07:23,881 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower799: fakeSnapshot purging log to 0 for term 6
01:07:23,881 PM [follower799-writer-173] [DEBUG] RaftStorageCompleter - follower799: 1 completion(s) pending
01:07:23,881 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: completed 1 commands in 3.508 ms
01:07:23,881 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: received 1 commands after 4.080 μs
01:07:23,881 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower799: running 1 completion(s)
01:07:23,881 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower799: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false}
01:07:23,881 PM [follower799-writer-173] [DEBUG] RaftStorageCompleter - follower799: 1 completion(s) pending
01:07:23,881 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorageCompleter - follower799: running 1 completion(s)
01:07:23,881 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: completed 1 commands in 150.4 μs
01:07:23,881 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: waiting to receive commands
01:07:24,69 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,189 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,309 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,429 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,669 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,789 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:24,910 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,29 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,150 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,270 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,389 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,509 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,629 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,749 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,869 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:25,989 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,99 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,219 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,339 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,459 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,579 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,699 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,819 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:26,929 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,49 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,159 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,269 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,379 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,499 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,619 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,739 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,859 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:27,979 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,99 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,219 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,339 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,459 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,579 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,699 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,820 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,928 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower799-collector801#-1118921336]
01:07:28,939 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower799: Lagging 0 entries behind leader leader798 and reached 1 (of expected -1)
01:07:28,939 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower799]
01:07:28,939 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower799: terminating thread VirtualThread[#965,follower799-writer-173]/waiting
01:07:28,939 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: received 1 commands after 5.058 s
01:07:28,940 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: completed 1 commands in 25.98 μs
01:07:28,940 PM [follower799-writer-173] [DEBUG] JournalWriteTask - follower799: journal writer stopped
01:07:28,940 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower799: journal closed
01:07:28,940 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower799: journal closed
01:07:28,940 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower799: stopped executor with 0 remaining tasks
01:07:28,940 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower799: stopped executor with 0 remaining tasks
01:07:28,940 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower800-collector802#-906778446]
01:07:28,940 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower800]
01:07:28,940 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower800: terminating thread VirtualThread[#966,follower800-writer-174]/waiting
01:07:28,940 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: received 1 commands after 5.662 s
01:07:28,941 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: completed 1 commands in 11.97 μs
01:07:28,941 PM [follower800-writer-174] [DEBUG] JournalWriteTask - follower800: journal writer stopped
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower800: journal closed
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower800: journal closed
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: stopped executor with 0 remaining tasks
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower800: stopped executor with 0 remaining tasks
01:07:28,941 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader798-collector803#-427420523]
01:07:28,941 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader798]
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - leader798: terminating thread VirtualThread[#967,leader798-writer-175]/waiting
01:07:28,941 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: received 1 commands after 5.060 s
01:07:28,941 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: completed 1 commands in 11.57 μs
01:07:28,941 PM [leader798-writer-175] [DEBUG] JournalWriteTask - leader798: journal writer stopped
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader798: journal closed
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader798: journal closed
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader798: stopped executor with 0 remaining tasks
01:07:28,941 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader798: stopped executor with 0 remaining tasks
01:07:28,945 PM [test-pekko.actor.default-dispatcher-16] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
[INFO] [07/01/2025 13:07:28.945] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
[WARN] [07/01/2025 13:07:28.946] [test-pekko.actor.default-dispatcher-18] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.171.139:17355)
[INFO] [07/01/2025 13:07:28.946] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - Exiting completed
01:07:28,946 PM [test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - Exiting completed
01:07:28,946 PM [test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - Shutting down...
01:07:28,947 PM [test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.139:17355] - Successfully shut down
[INFO] [07/01/2025 13:07:28.946] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - Shutting down...
[INFO] [07/01/2025 13:07:28.947] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.139:17355] - Successfully shut down
01:07:28,949 PM [test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.
01:07:28,949 PM [test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports.
[INFO] [07/01/2025 13:07:28.948] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.171.139:17355/system/remoting-terminator] Shutting down remote daemon.
[INFO] [07/01/2025 13:07:28.948] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.171.139:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports.