[DEBUG] [06/27/2025 14:32:12.300] [main] [EventStream(pekko://test)] logger log1-TestEventListener started 02:32:12,301 PM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started [DEBUG] [06/27/2025 14:32:12.301] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started [DEBUG] [06/27/2025 14:32:12.301] [main] [EventStream(pekko://test)] Default Loggers started [INFO] [06/27/2025 14:32:12.315] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.8:17355] with UID [1013553489405403648] 02:32:12,316 PM [test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.8:17355] with UID [1013553489405403648] 02:32:12,316 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Starting up, Pekko version [1.0.3] ... [INFO] [06/27/2025 14:32:12.316] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Starting up, Pekko version [1.0.3] ... 02:32:12,318 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Registered cluster JMX MBean [pekko:type=Cluster] [INFO] [06/27/2025 14:32:12.318] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 02:32:12,318 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Started up successfully [INFO] [06/27/2025 14:32:12.318] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Started up successfully 02:32:12,319 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 02:32:12,319 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining [INFO] [06/27/2025 14:32:12.319] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing [INFO] [06/27/2025 14:32:12.319] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 02:32:12,328 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries starting: sending 2 new payloads 02:32:12,330 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower760] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,333 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower760: Starting recovery with journal batch size 1000 [WARN] [06/27/2025 14:32:12.330] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower760]: org.opendaylight.controller.cluster.common.actor.Monitor@7c43f5fb [INFO] [pekkoDeadLetter][06/27/2025 14:32:12.330] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower760] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,333 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower760: started executor 02:32:12,333 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower760: started executor 02:32:12,334 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower760: metadata journal open: replayFrom=1 applyTo=0 02:32:12,334 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower760: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:12,334 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower760: journal open: applyTo=0 02:32:12,334 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@b06fb2a started 02:32:12,334 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting for more work 02:32:12,335 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/follower760/TermInfo.properties does not exist 02:32:12,335 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/follower760/journal-v1-1.log 02:32:12,335 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/follower760/metadata-v1-1.log 02:32:12,335 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: no eligible files found 02:32:12,335 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower760: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:12,340 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=0, votedFor=null 02:32:12,340 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower760: Local TermInfo store seeded with TermInfo{term=0} 02:32:12,340 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower760: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@8b2fb24 02:32:12,340 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from -1 to -1 02:32:12,341 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - follower760: journal recovery completedin 277.7 μs with journalIndex=1 [WARN] [06/27/2025 14:32:12.430] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower761]: org.opendaylight.controller.cluster.common.actor.Monitor@5f7929a [INFO] [pekkoDeadLetter][06/27/2025 14:32:12.431] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,431 PM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower761] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,431 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower761: Starting recovery with journal batch size 1000 02:32:12,431 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: started executor 02:32:12,432 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower761: started executor 02:32:12,432 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower761: metadata journal open: replayFrom=1 applyTo=0 02:32:12,433 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower761: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:12,433 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower761: journal open: applyTo=0 02:32:12,433 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@894c4e started 02:32:12,433 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: waiting for more work [WARN] [06/27/2025 14:32:12.434] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader759]: org.opendaylight.controller.cluster.common.actor.Monitor@17be6930 [INFO] [pekkoDeadLetter][06/27/2025 14:32:12.434] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader759] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,434 PM [test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader759] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,435 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/follower761/TermInfo.properties does not exist 02:32:12,435 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/follower761/journal-v1-1.log 02:32:12,435 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower761: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/follower761/metadata-v1-1.log 02:32:12,435 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower761: no eligible files found 02:32:12,435 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower761: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:12,435 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader759: Starting recovery with journal batch size 1000 02:32:12,436 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader759: started executor 02:32:12,436 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader759: started executor 02:32:12,436 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader759: metadata journal open: replayFrom=1 applyTo=0 02:32:12,437 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader759: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:12,437 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader759: journal open: applyTo=0 02:32:12,437 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@5b82f013 started 02:32:12,437 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting for more work 02:32:12,439 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=0, votedFor=null 02:32:12,439 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - follower761: Local TermInfo store seeded with TermInfo{term=0} 02:32:12,440 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower761: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@508fba6c 02:32:12,440 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to -1 02:32:12,440 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - follower761: journal recovery completedin 266.7 μs with journalIndex=1 02:32:12,440 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/leader759/TermInfo.properties does not exist 02:32:12,440 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader759: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/leader759/journal-v1-1.log 02:32:12,440 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader759: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest14622632486745655845/leader759/metadata-v1-1.log 02:32:12,440 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader759: no eligible files found 02:32:12,440 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=5, votedFor=leader759 02:32:12,440 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader759: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:12,443 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=5, votedFor=leader759 02:32:12,443 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader759: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader759} 02:32:12,443 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader759: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7574bb02 02:32:12,443 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from -1 to -1 02:32:12,443 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader759: journal recovery completedin 230.2 μs with journalIndex=1 02:32:12,926 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 02:32:12,927 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader759 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:12,927 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader759 (Follower): Received ElectionTimeout - switching to Candidate 02:32:12,927 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader759 (Candidate): Election: Candidate has following voting peers: [follower761, follower760] 02:32:12,931 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader759: Set currentTerm=6, votedFor=leader759 02:32:12,931 PM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader759 (Candidate): Starting new election term 6 02:32:12,931 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader759 (Candidate): Sending RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} to peer follower761 02:32:12,931 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader759 (Candidate): Sending RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} to peer follower760 02:32:12,931 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader759 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 02:32:12,931 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:12,931 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:12,931 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): Cluster state: [] 02:32:12,931 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:12,931 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Cluster state: [] 02:32:12,931 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower760 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:12,931 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:12,931 PM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower761 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:12,935 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=6, votedFor=null 02:32:12,935 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): In requestVote: RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:12,936 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=null 02:32:12,936 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): In requestVote: RequestVote{term=6, candidateId=leader759, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:12,938 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower760: Set currentTerm=6, votedFor=leader759 02:32:12,938 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower760 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:12,939 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader759 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 02:32:12,939 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader759 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 02:32:12,939 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): Election: Leader has following peers: [follower761, follower760] 02:32:12,939 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:12,939 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:12,939 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader759 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 02:32:12,939 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower760: Last sync leader does not match current leader leader759, need to catch up to -1 02:32:12,939 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: -1, nextIndex: 0 02:32:12,939 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:12,939 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower761: Set currentTerm=6, votedFor=leader759 02:32:12,939 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} [WARN] [06/27/2025 14:32:12.939] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower760] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower760, initialSyncDone=false] [INFO] [pekkoDeadLetter][06/27/2025 14:32:12.939] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower760] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,940 PM [test-pekko.actor.default-dispatcher-11] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,940 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower761: Last sync leader does not match current leader leader759, need to catch up to -1 02:32:12,940 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/27/2025 14:32:12.940] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=false] 02:32:12,940 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:12,940 PM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][06/27/2025 14:32:12.940] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,947 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader759: Persist data index=0 term=6 command=zero 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader759: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:12,947 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 entries in 509.8 ms 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower760 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower760: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Number of entries to be appended = 1 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Append entry to log zero 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower760: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Log size is now 1 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower760, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached -1 (of expected -1) 02:32:12,947 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower760: awaiting more completions to resolve 1 deferred callback(s) 02:32:12,947 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 entries in 613.1 ms 02:32:12,947 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: 0, nextIndex: 1 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader759: Persist data index=1 term=6 command=one 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader759: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 1 is present for follower follower760 02:32:12,948 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower760: AppendEntries{term=6, leaderId=leader759, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:12,953 PM [leader759-writer-162] [DEBUG] EntryStoreCompleter - leader759: 1 completion(s) pending 02:32:12,953 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: persisted 1 entries in 6.131 ms 02:32:12,953 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 entries in 6.770 μs 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader759: running 1 completion(s) 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader759: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader759 (Leader): Setting last applied to 0 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from -1 to 0 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader759: Applying state for log index 0 data zero 02:32:12,953 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader759: applyState called: zero 02:32:12,954 PM [follower760-writer-160] [DEBUG] EntryStoreCompleter - follower760: 1 completion(s) pending 02:32:12,954 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: persisted 1 entries in 6.732 ms 02:32:12,954 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting for more work 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower760: running 1 completion(s) 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower760: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower760: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@20e8e774 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Number of entries to be appended = 1 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Append entry to log one 02:32:12,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower760: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:12,955 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Log size is now 2 02:32:12,955 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower760, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:12,955 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 entries in 294.0 μs 02:32:12,955 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached -1 (of expected -1) 02:32:12,955 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower760: awaiting more completions to resolve 1 deferred callback(s) 02:32:12,955 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower760 updated: matchIndex: 1, nextIndex: 2 02:32:12,955 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower760: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:12,955 PM [leader759-writer-162] [DEBUG] EntryStoreCompleter - leader759: 1 completion(s) pending 02:32:12,955 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: persisted 1 entries in 2.157 ms 02:32:12,955 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 entries in 3.400 μs 02:32:12,955 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader759: running 1 completion(s) 02:32:12,955 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: persisted 1 entries in 79.29 μs 02:32:12,955 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting for more work 02:32:12,955 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader759: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:12,956 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:32:12,956 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader759 (Leader): Setting last applied to 1 02:32:12,956 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader759: Moving last applied index from 0 to 1 02:32:12,956 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader759: Applying state for log index 1 data one 02:32:12,956 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader759: applyState called: one 02:32:12,956 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 entries in 868.3 μs 02:32:12,956 PM [follower760-writer-160] [DEBUG] EntryStoreCompleter - follower760: 1 completion(s) pending 02:32:12,956 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: persisted 1 entries in 149.3 μs 02:32:12,956 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: waiting for more work 02:32:12,956 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: persisted 1 entries in 1.893 ms 02:32:12,956 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting for more work 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower760: running 1 completion(s) 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower760: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower760: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@1c15c70f 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Commit index set to 0 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 0 (of expected -1) 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): Setting last applied to 0 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from -1 to 0 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower760: Applying state for log index 0 data zero 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower760: applyState called: zero 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): Commit index set to 1 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower760 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower760 (Follower): Setting last applied to 1 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower760: Moving last applied index from 0 to 1 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower760: Applying state for log index 1 data one 02:32:12,957 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower760: applyState called: one 02:32:12,958 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 2 entries in 1.025 ms 02:32:12,958 PM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:12,958 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: persisted 2 entries in 328.4 μs 02:32:12,958 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: waiting for more work [WARN] [06/27/2025 14:32:12.957] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower760] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower760, initialSyncDone=true] [INFO] [pekkoDeadLetter][06/27/2025 14:32:12.958] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower760] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower760] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,57 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 02:32:13,57 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 02:32:13,176 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 02:32:13,176 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 02:32:13,176 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 02:32:13,198 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries: new entries applied - resuming follower follower761 02:32:13,296 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): sendAppendEntries: 0 is present for follower follower761 02:32:13,296 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Sending AppendEntries to follower follower761: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 02:32:13,296 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader759, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=false}, JournaledLogEntry{index=1, term=6, command=one, pending=false}]} 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): Number of entries to be appended = 2 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): Append entry to log zero 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): Append entry to log one 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower761: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): Log size is now 2 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): Commit index set to 1 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower761, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower761 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 0 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from -1 to 0 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower761: Applying state for log index 0 data zero 02:32:13,297 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower761: applyState called: zero 02:32:13,297 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 2 entries in 864.0 ms 02:32:13,297 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower761 updated: matchIndex: 1, nextIndex: 2 02:32:13,297 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader759: fakeSnapshot purging log to 0 for term 6 02:32:13,297 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader759 (Leader): Checking sendAppendEntries for follower follower761: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:32:13,298 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower761 (Follower): Setting last applied to 1 02:32:13,298 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower761: Moving last applied index from 0 to 1 02:32:13,298 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower761: Applying state for log index 1 data one 02:32:13,298 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower761: applyState called: one 02:32:13,298 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower761: awaiting more completions to resolve 2 deferred callback(s) 02:32:13,304 PM [follower761-writer-161] [DEBUG] EntryStoreCompleter - follower761: 2 completion(s) pending 02:32:13,304 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower761: running 2 completion(s) 02:32:13,304 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: persisted 2 entries in 6.842 ms 02:32:13,304 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:13,304 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 1 entries in 6.140 μs 02:32:13,304 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower761: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@6797c0a 02:32:13,304 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower761: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:13,304 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower761: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@22bce523 02:32:13,304 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: persisted 1 entries in 87.22 μs 02:32:13,304 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: waiting for more work [WARN] [06/27/2025 14:32:13.304] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower761] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower761, initialSyncDone=true] [INFO] [pekkoDeadLetter][06/27/2025 14:32:13.304] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower761] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,304 PM [test-pekko.actor.default-dispatcher-16] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower761] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,417 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower760: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 02:32:13,417 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower761: Lagging 0 entries behind leader leader759 and reached 1 (of expected -1) 02:32:13,417 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - follower760: fakeSnapshot purging log to 0 for term 6 02:32:13,417 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower761: fakeSnapshot purging log to 0 for term 6 02:32:13,450 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testReplicationsWithLaggingFollowerCaughtUpViaAppendEntries complete 02:32:13,450 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower760-collector762#-1231799870] 02:32:13,461 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower760] 02:32:13,462 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - follower760: terminating thread VirtualThread[#893,follower760-writer-160]/waiting 02:32:13,462 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: received 1 entries in 503.7 ms 02:32:13,462 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: persisted 1 entries in 31.08 μs 02:32:13,462 PM [follower760-writer-160] [DEBUG] JournalWriteTask - follower760: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@b06fb2a stopped 02:32:13,462 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower760: journal closed 02:32:13,462 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower760: journal closed 02:32:13,462 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: stopped executor with 0 remaining tasks 02:32:13,462 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower760: stopped executor with 0 remaining tasks 02:32:13,462 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower761-collector763#588009885] 02:32:13,462 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower761] 02:32:13,462 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - follower761: terminating thread VirtualThread[#894,follower761-writer-161]/waiting 02:32:13,463 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: received 1 entries in 158.5 ms 02:32:13,463 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: persisted 1 entries in 17.25 μs 02:32:13,463 PM [follower761-writer-161] [DEBUG] JournalWriteTask - follower761: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@894c4e stopped 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower761: journal closed 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower761: journal closed 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower761: stopped executor with 0 remaining tasks 02:32:13,463 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader759-collector764#617838222] 02:32:13,463 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader759] 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - leader759: terminating thread VirtualThread[#895,leader759-writer-162]/waiting 02:32:13,463 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: received 1 entries in 506.8 ms 02:32:13,463 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: persisted 1 entries in 17.78 μs 02:32:13,463 PM [leader759-writer-162] [DEBUG] JournalWriteTask - leader759: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@5b82f013 stopped 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader759: journal closed 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader759: journal closed 02:32:13,463 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: stopped executor with 0 remaining tasks 02:32:13,464 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader759: stopped executor with 0 remaining tasks 02:32:13,465 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaInstallSnapshot starting 02:32:13,467 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower769] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [WARN] [06/27/2025 14:32:13.468] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower769]: org.opendaylight.controller.cluster.common.actor.Monitor@27e78345 [INFO] [pekkoDeadLetter][06/27/2025 14:32:13.467] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower769] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,468 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - follower769: Starting recovery with journal batch size 1000 02:32:13,468 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower769: started executor 02:32:13,468 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower769: started executor 02:32:13,469 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower769: metadata journal open: replayFrom=1 applyTo=0 02:32:13,470 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower769: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:13,470 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower769: journal open: applyTo=0 02:32:13,470 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@2e79b3ac started 02:32:13,470 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:13,471 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/follower769/TermInfo.properties does not exist [WARN] [06/27/2025 14:32:13.471] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower770]: org.opendaylight.controller.cluster.common.actor.Monitor@7719c8e6 02:32:13,471 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/follower769/journal-v1-1.log 02:32:13,471 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower769: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/follower769/metadata-v1-1.log 02:32:13,472 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower769: no eligible files found [INFO] [pekkoDeadLetter][06/27/2025 14:32:13.472] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,472 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower769: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:13,472 PM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower770] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,472 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower770: Starting recovery with journal batch size 1000 02:32:13,473 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower770: started executor 02:32:13,473 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower770: started executor 02:32:13,474 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower770: metadata journal open: replayFrom=1 applyTo=0 02:32:13,474 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower770: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:13,474 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower770: journal open: applyTo=0 02:32:13,474 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@39845ddd started 02:32:13,474 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting for more work 02:32:13,475 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=0, votedFor=null 02:32:13,475 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower769: Local TermInfo store seeded with TermInfo{term=0} 02:32:13,475 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@297a80c5 02:32:13,475 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from -1 to -1 02:32:13,475 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] JournalRecovery - follower769: journal recovery completedin 192.7 μs with journalIndex=1 [WARN] [06/27/2025 14:32:13.476] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@2f1e2cbd [INFO] [pekkoDeadLetter][06/27/2025 14:32:13.476] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader768] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,476 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader768] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 02:32:13,476 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/follower770/TermInfo.properties does not exist 02:32:13,476 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 02:32:13,477 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/follower770/journal-v1-1.log 02:32:13,477 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower770: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/follower770/metadata-v1-1.log 02:32:13,477 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower770: no eligible files found 02:32:13,477 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: started executor 02:32:13,477 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower770: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:13,477 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: started executor 02:32:13,477 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=0 02:32:13,478 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:13,478 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=0 02:32:13,478 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@2f449ad3 started 02:32:13,478 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:13,479 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties does not exist 02:32:13,480 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:13,480 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:13,480 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:13,480 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 02:32:13,480 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:13,481 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=0, votedFor=null 02:32:13,481 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower770: Local TermInfo store seeded with TermInfo{term=0} 02:32:13,481 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower770: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@6b02b7e6 02:32:13,481 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to -1 02:32:13,482 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - follower770: journal recovery completedin 370.7 μs with journalIndex=1 02:32:13,482 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 02:32:13,482 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader768: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader768} 02:32:13,482 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@26301e51 02:32:13,482 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 02:32:13,482 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: journal recovery completedin 145.8 μs with journalIndex=1 02:32:13,966 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 02:32:13,966 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:13,967 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 02:32:13,967 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 02:32:13,970 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 02:32:13,970 PM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader768 (Candidate): Starting new election term 6 02:32:13,970 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} to peer follower769 02:32:13,970 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} to peer follower770 02:32:13,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:13,970 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 02:32:13,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 02:32:13,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:13,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:13,970 PM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower769 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:13,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 02:32:13,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:13,970 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower770 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:13,973 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=6, votedFor=null 02:32:13,973 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:13,974 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=null 02:32:13,974 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=6, candidateId=leader768, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:13,977 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=6, votedFor=leader768 02:32:13,977 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:13,977 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 02:32:13,977 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 02:32:13,977 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Election: Leader has following peers: [follower769, follower770] 02:32:13,977 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:13,977 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:13,977 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 02:32:13,978 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=6, votedFor=leader768 02:32:13,978 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Last sync leader does not match current leader leader768, need to catch up to -1 02:32:13,978 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:13,978 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Last sync leader does not match current leader leader768, need to catch up to -1 [WARN] [06/27/2025 14:32:13.978] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower769] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower769, initialSyncDone=false] 02:32:13,978 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: -1, nextIndex: 0 02:32:13,978 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/27/2025 14:32:13.978] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=false] 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:13,979 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=0 term=6 command=zero 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower769 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:13,979 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 500.7 ms 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower770 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader768: Persist data index=1 term=6 command=one 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower769 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 0 is present for follower follower770 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:13,979 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log zero 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1 02:32:13,979 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:13,979 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Append entry to log zero 02:32:13,979 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:13,979 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Log size is now 1 02:32:13,980 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:13,980 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 509.7 ms 02:32:13,980 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 02:32:13,980 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 0, nextIndex: 1 02:32:13,980 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Log size is now 1 02:32:13,980 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:13,980 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 entries in 505.3 ms 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 1 is present for follower follower769 02:32:13,980 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:13,980 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: awaiting more completions to resolve 1 deferred callback(s) 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 0, nextIndex: 1 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 1 is present for follower follower770 02:32:13,980 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower770: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:13,984 PM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 02:32:13,984 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 02:32:13,984 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 5.221 ms 02:32:13,984 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:13,984 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 5.920 μs 02:32:13,984 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:32:13,984 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 0 02:32:13,985 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to 0 02:32:13,985 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Applying state for log index 0 data zero 02:32:13,985 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader768: applyState called: zero 02:32:13,985 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 02:32:13,985 PM [follower770-writer-164] [DEBUG] EntryStoreCompleter - follower770: 1 completion(s) pending 02:32:13,985 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 02:32:13,985 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: persisted 1 entries in 5.667 ms 02:32:13,985 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting for more work 02:32:13,985 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 5.913 ms 02:32:13,985 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:13,985 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:13,985 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@51020321 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: running 1 completion(s) 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@31534c84 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Number of entries to be appended = 1 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log one 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Append entry to log one 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Log size is now 2 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:13,986 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 entries in 264.8 μs 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 02:32:13,986 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 259.4 μs 02:32:13,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 1, nextIndex: 2 02:32:13,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Log size is now 2 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower770, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:13,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached -1 (of expected -1) 02:32:13,986 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: awaiting more completions to resolve 1 deferred callback(s) 02:32:13,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower770 updated: matchIndex: 1, nextIndex: 2 02:32:13,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:13,986 PM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 02:32:13,986 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 2.013 ms 02:32:13,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 02:32:13,986 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 7.120 μs 02:32:13,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:13,987 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:32:13,987 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 83.91 μs 02:32:13,987 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:13,987 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 1 02:32:13,987 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 0 to 1 02:32:13,987 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader768: Applying state for log index 1 data one 02:32:13,987 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader768: applyState called: one 02:32:13,987 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - leader768: fakeSnapshot purging log to 0 for term 6 02:32:13,987 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 225.9 μs 02:32:13,987 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 53.06 μs 02:32:13,987 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:13,987 PM [follower770-writer-164] [DEBUG] EntryStoreCompleter - follower770: 1 completion(s) pending 02:32:13,987 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 02:32:13,987 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: persisted 1 entries in 1.591 ms 02:32:13,987 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting for more work 02:32:13,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 02:32:13,987 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 1.586 ms 02:32:13,987 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:13,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:13,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@74369be2 02:32:13,987 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: running 1 completion(s) 02:32:13,987 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower770: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:13,987 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower770: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@59899f85 02:32:13,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Commit index set to 0 02:32:13,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 0 (of expected -1) 02:32:13,987 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): Commit index set to 0 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 0 (of expected -1) 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 0 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from -1 to 0 02:32:13,988 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower769: Applying state for log index 0 data zero 02:32:13,988 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 0 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower769: applyState called: zero 02:32:13,988 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from -1 to 0 02:32:13,988 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower770: Applying state for log index 0 data zero 02:32:13,988 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower770: applyState called: zero 02:32:13,988 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 entries in 291.7 μs 02:32:13,988 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 272.0 μs 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Commit index set to 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:13,988 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: persisted 1 entries in 51.48 μs 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:13,988 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting for more work 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 0 to 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower769: Applying state for log index 1 data one 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower769: applyState called: one 02:32:13,988 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 77.00 μs 02:32:13,988 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): Commit index set to 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower770 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower770 (Follower): Setting last applied to 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower770: Moving last applied index from 0 to 1 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower770: Applying state for log index 1 data one 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower770: applyState called: one 02:32:13,988 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower770: fakeSnapshot purging log to 0 for term 6 02:32:13,988 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 entries in 298.3 μs 02:32:13,988 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: persisted 1 entries in 70.91 μs 02:32:13,988 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: waiting for more work [WARN] [06/27/2025 14:32:13.988] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower770] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower770, initialSyncDone=true] 02:32:13,988 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower769: fakeSnapshot purging log to 0 for term 6 02:32:13,988 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 578.9 μs 02:32:13,988 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 67.80 μs 02:32:13,988 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work [WARN] [06/27/2025 14:32:13.989] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower769] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower769, initialSyncDone=true] 02:32:14,97 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower770: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,97 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,216 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,336 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,457 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=2 term=6 command=two 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 02:32:14,536 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 548.9 ms 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=3 term=6 command=three 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=4 term=6 command=four 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=5 term=6 command=five 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log two 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=6 term=6 command=six 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Log size is now 2 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 02:32:14,536 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:14,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 2 is present for follower follower769 02:32:14,536 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 548.0 ms 02:32:14,537 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 02:32:14,537 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 02:32:14,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 2, nextIndex: 3 02:32:14,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 0 02:32:14,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): sendAppendEntries: 3 is present for follower follower769 02:32:14,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Sending AppendEntries to follower follower769: AppendEntries{term=6, leaderId=leader768, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 02:32:14,539 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 02:32:14,539 PM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 02:32:14,539 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 2.334 ms 02:32:14,539 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 3.041 ms 02:32:14,539 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 02:32:14,539 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 4 entries in 5.850 μs 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@35655400 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader768 (Leader): Setting last applied to 2 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 1 to 2 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Applying state for log index 2 data two 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader768: applyState called: two 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader768, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}, JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 02:32:14,539 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SnapshotManager - leader768: fakeSnapshot purging log to 1 for term 6 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 4 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log three 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log four 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log five 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 02:32:14,539 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 539.2 μs 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Append entry to log six 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 02:32:14,539 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Log size is now 6 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower769, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 1 (of expected -1) 02:32:14,540 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 6, nextIndex: 7 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 4 deferred callback(s) 02:32:14,540 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1 02:32:14,540 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 02:32:14,540 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 871.8 μs 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 02:32:14,540 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 3 entries in 5.351 μs 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@2e1ca823 02:32:14,540 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 3 deferred callback(s) 02:32:14,543 PM [leader768-writer-165] [DEBUG] EntryStoreCompleter - leader768: 4 completion(s) pending 02:32:14,543 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 4 entries in 4.146 ms 02:32:14,543 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader768: running 4 completion(s) 02:32:14,543 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 5.570 μs 02:32:14,543 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:14,543 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 99.02 μs 02:32:14,543 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:14,543 PM [test-pekko.actor.default-dispatcher-5] [INFO] SnapshotManager - leader768: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=4, mandatoryTrim=false] 02:32:14,544 PM [test-pekko.actor.default-dispatcher-16] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [ERROR] [06/27/2025 14:32:14.544] [test-pekko.actor.internal-dispatcher-2] [pekko://test/user/leader768] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:32:14,544 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 3 completion(s) pending 02:32:14,544 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 3 entries in 3.975 ms 02:32:14,544 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:14,544 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: running 3 completion(s) 02:32:14,544 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false} 02:32:14,544 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@11d228eb 02:32:14,544 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false} 02:32:14,544 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@69b2196 02:32:14,544 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#900,leader768-writer-165]/waiting 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false} 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@5a4cae29 02:32:14,545 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: received 1 entries in 1.267 ms 02:32:14,545 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 25.45 μs 02:32:14,545 PM [leader768-writer-165] [DEBUG] JournalWriteTask - leader768: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@2f449ad3 stopped 02:32:14,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: journal closed 02:32:14,545 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader768: journal closed 02:32:14,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 02:32:14,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): Commit index set to 2 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 2 (of expected -1) 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 6 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 2 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 1 to 2 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower769: Applying state for log index 2 data two 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower769: applyState called: two 02:32:14,545 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower769: fakeSnapshot purging log to 1 for term 6 02:32:14,545 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 977.1 μs [WARN] [06/27/2025 14:32:14.545] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@2d244c2c 02:32:14,546 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 138.8 μs 02:32:14,546 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:14,546 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 02:32:14,546 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: started executor 02:32:14,546 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: started executor 02:32:14,547 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=2 02:32:14,548 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=7 replayFrom=1 fileEntries=0 02:32:14,548 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=2 02:32:14,548 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6f98bc93 started 02:32:14,548 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:14,549 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 02:32:14,549 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:14,549 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader768: restoring local TermInfo{term=6, votedFor=leader768} 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=6, votedFor=leader768 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@59847e01 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 02:32:14,550 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=1, term=6, command=one] 02:32:14,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=2, term=6, command=two] 02:32:14,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=3, term=6, command=three] 02:32:14,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=4, term=6, command=four] 02:32:14,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=5, term=6, command=five] 02:32:14,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=6, term=6, command=six] 02:32:14,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader768: journal recovery completedin 1.865 ms with journalIndex=1 02:32:14,636 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:14,636 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:14,637 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:14,637 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:14,737 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:14,737 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:14,737 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:14,737 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:14,838 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:14,838 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:14,838 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:14,838 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:14,938 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:14,938 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:14,938 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:14,938 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,39 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,39 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,39 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,39 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,66 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:15,66 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 02:32:15,66 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 02:32:15,70 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 02:32:15,70 PM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader768 (Candidate): Starting new election term 7 02:32:15,70 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} to peer follower769 02:32:15,70 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} to peer follower770 02:32:15,70 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 02:32:15,70 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:15,70 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:15,70 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 02:32:15,70 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 02:32:15,70 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 02:32:15,70 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 02:32:15,70 PM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower769 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 02:32:15,70 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower770 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term 02:32:15,74 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=7, votedFor=null 02:32:15,74 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 6 02:32:15,75 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=7, votedFor=null 02:32:15,75 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=7, candidateId=leader768, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 6, lastTerm: 6 02:32:15,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=7, votedFor=leader768 02:32:15,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader768 (Candidate): LastApplied index -1 is behind last index 6 - switching to PreLeader 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Election: Leader has following peers: [follower769, follower770] 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 02:32:15,77 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=7, votedFor=leader768 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Persist data index=7 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,77 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 02:32:15,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Replicate message: identifier: null, logIndex: 7, isSendImmediate: true 02:32:15,78 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:15,78 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: received 1 entries in 529.8 ms 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:15,78 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 6, nextIndex: 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 7 is present for follower follower769 02:32:15,78 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Sending AppendEntries to follower follower769: AppendEntries{term=7, leaderId=leader768, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 7 is present for follower follower769 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader768, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): Log size is now 6 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower769, logLastIndex=7, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 02:32:15,79 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 533.5 ms 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 7, nextIndex: 8 02:32:15,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 02:32:15,79 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:15,80 PM [leader768-writer-166] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 02:32:15,80 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 1.773 ms 02:32:15,80 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:15,80 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 02:32:15,80 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:15,80 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader768: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=7, lastTerm=7, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=8, mandatoryTrim=false] 02:32:15,80 PM [test-pekko.actor.default-dispatcher-5] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:32:15,80 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#901,leader768-writer-166]/waiting 02:32:15,81 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: received 1 entries in 906.5 μs 02:32:15,81 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 19.28 μs 02:32:15,81 PM [leader768-writer-166] [DEBUG] JournalWriteTask - leader768: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6f98bc93 stopped [ERROR] [06/27/2025 14:32:15.080] [test-pekko.actor.internal-dispatcher-4] [pekko://test/user/leader768] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:32:15,81 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 02:32:15,81 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 02:32:15,81 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 1.692 ms 02:32:15,81 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:15,81 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=false} 02:32:15,81 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@68e0542c 02:32:15,81 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader768: journal closed 02:32:15,81 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader768: journal closed 02:32:15,81 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 02:32:15,81 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks [WARN] [06/27/2025 14:32:15.082] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader768]: org.opendaylight.controller.cluster.common.actor.Monitor@52087d9d 02:32:15,82 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - leader768: Starting recovery with journal batch size 1000 02:32:15,82 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader768: started executor 02:32:15,83 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader768: started executor 02:32:15,83 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader768: metadata journal open: replayFrom=1 applyTo=2 02:32:15,85 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader768: journal open: firstIndex=1 lastIndex=8 replayFrom=1 fileEntries=0 02:32:15,85 PM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader768: journal open: applyTo=2 02:32:15,85 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1c456a86 started 02:32:15,85 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:15,86 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 02:32:15,86 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,86 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,86 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=5, votedFor=leader768 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRecovery - leader768: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRecovery - leader768: restoring local TermInfo{term=7, votedFor=leader768} 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=7, votedFor=leader768 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader768: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@5a7034ea 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to -1 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=1, term=6, command=one] 02:32:15,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=2, term=6, command=two] 02:32:15,88 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=3, term=6, command=three] 02:32:15,88 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=4, term=6, command=four] 02:32:15,88 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=5, term=6, command=five] 02:32:15,88 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=6, term=6, command=six] 02:32:15,89 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: recovered journal DefaultLogEntry[index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7] 02:32:15,89 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] JournalRecovery - leader768: journal recovery completedin 2.312 ms with journalIndex=1 02:32:15,139 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,139 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,139 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,139 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,240 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,240 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,240 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,240 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,341 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,341 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,341 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,341 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,441 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,441 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,441 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,441 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,542 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,542 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,542 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,542 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,567 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader768 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:15,567 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader768 (Follower): Received ElectionTimeout - switching to Candidate 02:32:15,567 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader768 (Candidate): Election: Candidate has following voting peers: [follower769, follower770] 02:32:15,571 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader768: Set currentTerm=8, votedFor=leader768 02:32:15,571 PM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader768 (Candidate): Starting new election term 8 02:32:15,571 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} to peer follower769 02:32:15,571 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader768 (Candidate): Sending RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} to peer follower770 02:32:15,571 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader768 (Follower) :- Switching from behavior Follower to Candidate, election term: 8 02:32:15,572 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower770 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:15,572 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower770 (Follower): Cluster state: [] 02:32:15,572 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower770 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 02:32:15,572 PM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower770 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 02:32:15,572 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:15,572 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Cluster state: [] 02:32:15,572 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 02:32:15,572 PM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower769 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 02:32:15,575 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=8, votedFor=null 02:32:15,575 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower770 (Follower): In requestVote: RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 6 02:32:15,577 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=8, votedFor=null 02:32:15,577 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): In requestVote: RequestVote{term=8, candidateId=leader768, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 7, lastTerm: 7 02:32:15,578 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower770: Set currentTerm=8, votedFor=leader768 02:32:15,578 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower770 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 02:32:15,578 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader768 (Candidate): RaftRPC message received RequestVoteReply{term=8, voteGranted=true}, my term is 8 02:32:15,578 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader768 (Candidate): handleRequestVoteReply: RequestVoteReply{term=8, voteGranted=true}, current voteCount: 1 02:32:15,578 PM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader768 (Candidate): LastApplied index -1 is behind last index 7 - switching to PreLeader 02:32:15,578 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Election: Leader has following peers: [follower769, follower770] 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader768 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader768: Persist data index=8 term=8 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader768: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:15,579 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 entries in 493.6 ms 02:32:15,579 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower769: Set currentTerm=8, votedFor=leader768 02:32:15,579 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower769 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 02:32:15,579 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Replicate message: identifier: null, logIndex: 8, isSendImmediate: true 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 7, nextIndex: 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 8 is present for follower follower769 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader768 (PreLeader): Sending AppendEntries to follower follower769: AppendEntries{term=8, leaderId=leader768, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 02:32:15,579 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): handleAppendEntries: AppendEntries{term=8, leaderId=leader768, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): Number of entries to be appended = 1 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower769: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): Log size is now 7 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): handleAppendEntries returning : AppendEntriesReply{term=8, success=true, followerId=follower769, logLastIndex=8, logLastTerm=8, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging -3 entries behind leader leader768 and reached 2 (of expected -1) 02:32:15,580 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower769: awaiting more completions to resolve 1 deferred callback(s) 02:32:15,580 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 498.8 ms 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): sendAppendEntries: 8 is present for follower follower769 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower769 updated: matchIndex: 8, nextIndex: 9 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:15,580 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader768 (PreLeader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:15,580 PM [leader768-writer-167] [DEBUG] EntryStoreCompleter - leader768: 1 completion(s) pending 02:32:15,580 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 1.451 ms 02:32:15,580 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:15,580 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader768: running 1 completion(s) 02:32:15,580 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader768: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:15,580 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (PreLeader): Applying to log - commitIndex: 8, lastAppliedIndex: -1 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 0 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from -1 to 0 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 0 data zero 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: zero 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 1 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 0 to 1 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 1 data one 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: one 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 2 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 1 to 2 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 2 data two 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: two 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 3 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 2 to 3 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 3 data three 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: three 02:32:15,581 PM [follower769-writer-163] [DEBUG] EntryStoreCompleter - follower769: 1 completion(s) pending 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 4 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 3 to 4 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 4 data four 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: four 02:32:15,581 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower769: running 1 completion(s) 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 5 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 4 to 5 02:32:15,581 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 1.453 ms 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 5 data five 02:32:15,581 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: five 02:32:15,581 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower769: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=false} 02:32:15,581 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower769: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@1b9522b 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 6 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 5 to 6 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 6 data six 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader768: applyState called: six 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 7 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 6 to 7 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader768 (PreLeader): Setting last applied to 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader768: Moving last applied index from 7 to 8 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader768: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,581 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 entries in 918.2 μs 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PreLeader - leader768 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 8, lastIndex: 8 02:32:15,581 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 71.70 μs 02:32:15,581 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: waiting for more work 02:32:15,581 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (Leader): Election: Leader has following peers: [follower769, follower770] 02:32:15,582 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower769: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:15,582 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader768 (Leader): Checking sendAppendEntries for follower follower770: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:15,582 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader768 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 8 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): Commit index set to 8 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower769 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 2, lastIndex(): 8 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 3 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 2 to 3 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Applying state for log index 3 data three 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower769: applyState called: three 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 4 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 3 to 4 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Applying state for log index 4 data four 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower769: applyState called: four 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 5 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 4 to 5 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Applying state for log index 5 data five 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower769: applyState called: five 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 6 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 5 to 6 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Applying state for log index 6 data six 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower769: applyState called: six 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 7 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 6 to 7 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower769 (Follower): Setting last applied to 8 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower769: Moving last applied index from 7 to 8 02:32:15,582 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower769: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:15,582 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 884.5 μs 02:32:15,582 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 51.61 μs 02:32:15,582 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: waiting for more work 02:32:15,642 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,642 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,642 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,642 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,697 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:15,743 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,743 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,743 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,743 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,817 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:15,844 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,844 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,844 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,844 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:15,937 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:15,944 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:15,944 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:15,944 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:15,944 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,45 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,45 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,45 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,45 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,57 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,146 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,146 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,146 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,146 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,177 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,246 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,246 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,246 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,246 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,296 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,347 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,347 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,347 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,347 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,417 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,448 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,448 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,448 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,448 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,536 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,548 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,548 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,548 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,548 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,649 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,649 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,649 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,649 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,656 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,749 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,750 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,750 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,751 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,777 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,851 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,851 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,851 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,851 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:16,897 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:16,952 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:16,952 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:16,952 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:16,952 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,16 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,53 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,53 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,53 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,53 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,137 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,153 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,154 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,154 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,154 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,254 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,254 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,254 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,254 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,257 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,355 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,355 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,355 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,355 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,376 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,456 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,456 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,456 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,456 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,497 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,556 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,557 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,557 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,557 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,617 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,657 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,658 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,658 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,658 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,737 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,758 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,758 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,758 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,758 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,857 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:17,859 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,859 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,859 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,859 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,960 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:17,960 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:17,960 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:17,960 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:17,977 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,60 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,60 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,61 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,61 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,98 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,161 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,162 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,162 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,162 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,217 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,262 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,262 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,262 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,262 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,337 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,363 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,363 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,363 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,363 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,456 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,464 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,464 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,464 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,464 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,565 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,565 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,565 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,565 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,577 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,666 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,666 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,666 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,666 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,697 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,767 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,767 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,767 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,767 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,817 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,867 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,867 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,867 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,867 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:18,937 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:18,968 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:18,968 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:18,968 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:18,968 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:19,57 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:19,69 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:19,69 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:19,69 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:19,69 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:19,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:19,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:19,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:19,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:19,176 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:19,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:19,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:19,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:19,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:19,297 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:19,371 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:19,371 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:19,371 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:19,371 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:19,417 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:19,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/TermInfo.properties 02:32:19,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/journal-v1-1.log 02:32:19,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12239842041403670502/leader768/metadata-v1-1.log 02:32:19,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader768: no eligible files found 02:32:19,537 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower769: Lagging 0 entries behind leader leader768 and reached 8 (of expected -1) 02:32:19,574 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower769-collector771#-1267704002] 02:32:19,585 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower769] 02:32:19,586 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower769: terminating thread VirtualThread[#898,follower769-writer-163]/waiting 02:32:19,586 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: received 1 entries in 4.004 s 02:32:19,586 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: persisted 1 entries in 44.80 μs 02:32:19,586 PM [follower769-writer-163] [DEBUG] JournalWriteTask - follower769: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@2e79b3ac stopped 02:32:19,586 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower769: journal closed 02:32:19,586 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower769: journal closed 02:32:19,586 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower769: stopped executor with 0 remaining tasks 02:32:19,586 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower769: stopped executor with 0 remaining tasks 02:32:19,587 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower770-collector772#-1867372085] 02:32:19,587 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower770] 02:32:19,587 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower770: terminating thread VirtualThread[#899,follower770-writer-164]/waiting 02:32:19,587 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: received 1 entries in 5.599 s 02:32:19,587 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: persisted 1 entries in 18.70 μs 02:32:19,587 PM [follower770-writer-164] [DEBUG] JournalWriteTask - follower770: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@39845ddd stopped 02:32:19,587 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower770: journal closed 02:32:19,587 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower770: journal closed 02:32:19,587 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks 02:32:19,587 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower770: stopped executor with 0 remaining tasks 02:32:19,587 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader768-collector773#1864110572] 02:32:19,587 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader768] 02:32:19,588 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader768: terminating thread VirtualThread[#903,leader768-writer-167]/waiting 02:32:19,588 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: received 1 entries in 4.006 s 02:32:19,588 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: persisted 1 entries in 16.37 μs 02:32:19,588 PM [leader768-writer-167] [DEBUG] JournalWriteTask - leader768: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1c456a86 stopped 02:32:19,588 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader768: journal closed 02:32:19,588 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader768: journal closed 02:32:19,588 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 02:32:19,588 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader768: stopped executor with 0 remaining tasks 02:32:19,591 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotWithLaggingFollowerCaughtUpViaAppendEntries starting [WARN] [06/27/2025 14:32:19.593] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower778]: org.opendaylight.controller.cluster.common.actor.Monitor@2894fdad 02:32:19,594 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - follower778: Starting recovery with journal batch size 1000 02:32:19,594 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: started executor 02:32:19,594 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower778: started executor 02:32:19,595 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower778: metadata journal open: replayFrom=1 applyTo=0 02:32:19,595 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower778: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:19,595 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower778: journal open: applyTo=0 02:32:19,595 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@148ed076 started 02:32:19,595 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:19,596 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/follower778/TermInfo.properties does not exist 02:32:19,597 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/follower778/journal-v1-1.log 02:32:19,597 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower778: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/follower778/metadata-v1-1.log 02:32:19,597 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower778: no eligible files found 02:32:19,597 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower778: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/27/2025 14:32:19.598] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower779]: org.opendaylight.controller.cluster.common.actor.Monitor@d80f5c9 02:32:19,599 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - follower779: Starting recovery with journal batch size 1000 02:32:19,599 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: started executor 02:32:19,599 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: started executor 02:32:19,600 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower779: metadata journal open: replayFrom=1 applyTo=0 02:32:19,600 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower779: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:19,600 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - follower779: journal open: applyTo=0 02:32:19,600 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1774b09a started 02:32:19,600 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting for more work 02:32:19,601 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=0, votedFor=null 02:32:19,601 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower778: Local TermInfo store seeded with TermInfo{term=0} 02:32:19,601 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower778: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@40e42f6c 02:32:19,601 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from -1 to -1 02:32:19,601 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/follower779/TermInfo.properties does not exist 02:32:19,601 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/follower779/journal-v1-1.log 02:32:19,601 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower778: journal recovery completedin 539.9 μs with journalIndex=1 02:32:19,601 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/follower779/metadata-v1-1.log 02:32:19,601 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower779: no eligible files found 02:32:19,601 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower779: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/27/2025 14:32:19.602] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader777]: org.opendaylight.controller.cluster.common.actor.Monitor@3815a597 02:32:19,603 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - leader777: Starting recovery with journal batch size 1000 02:32:19,603 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: started executor 02:32:19,603 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader777: started executor 02:32:19,603 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: metadata journal open: replayFrom=1 applyTo=0 02:32:19,604 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader777: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:19,604 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader777: journal open: applyTo=0 02:32:19,604 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@cb7b73c started 02:32:19,604 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:19,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=0, votedFor=null 02:32:19,605 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower779: Local TermInfo store seeded with TermInfo{term=0} 02:32:19,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower779: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@381b336b 02:32:19,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to -1 02:32:19,605 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties does not exist 02:32:19,605 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:19,605 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:19,605 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:19,605 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 02:32:19,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - follower779: journal recovery completedin 182.7 μs with journalIndex=1 02:32:19,605 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader777: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:19,608 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 02:32:19,608 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader777: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader777} 02:32:19,608 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2277b604 02:32:19,608 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to -1 02:32:19,608 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] JournalRecovery - leader777: journal recovery completedin 191.6 μs with journalIndex=1 02:32:20,66 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 02:32:20,66 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader777 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:20,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader777 (Follower): Received ElectionTimeout - switching to Candidate 02:32:20,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): Election: Candidate has following voting peers: [follower779, follower778] 02:32:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 02:32:20,71 PM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader777 (Candidate): Starting new election term 6 02:32:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} to peer follower779 02:32:20,71 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} to peer follower778 02:32:20,71 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader777 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 02:32:20,72 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:20,72 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: [] 02:32:20,72 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:20,72 PM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower779 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:20,72 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:20,72 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Cluster state: [] 02:32:20,72 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:20,72 PM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower778 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:20,75 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=null 02:32:20,75 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:20,81 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=6, votedFor=null 02:32:20,81 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): In requestVote: RequestVote{term=6, candidateId=leader777, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=6, votedFor=leader777 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader777 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Election: Leader has following peers: [follower779, follower778] 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader777 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 02:32:20,83 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower779: Last sync leader does not match current leader leader777, need to catch up to -1 02:32:20,84 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: -1, nextIndex: 0 02:32:20,84 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/27/2025 14:32:20.084] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=false] 02:32:20,86 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=6, votedFor=leader777 02:32:20,86 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:20,86 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Last sync leader does not match current leader leader777, need to catch up to -1 02:32:20,86 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/27/2025 14:32:20.086] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower778] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower778, initialSyncDone=false] 02:32:20,86 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:20,113 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 02:32:20,113 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader777: Persist data index=0 term=6 command=zero 02:32:20,113 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:20,113 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 02:32:20,113 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:20,113 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower779 02:32:20,113 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 509.7 ms 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower778 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader777: Persist data index=1 term=6 command=one 02:32:20,114 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 02:32:20,114 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:20,114 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Append entry to log zero 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower779 02:32:20,114 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 0 is present for follower follower778 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Append entry to log zero 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Log size is now 1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 02:32:20,114 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 entries in 513.8 ms 02:32:20,114 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 518.9 ms 02:32:20,114 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 0, nextIndex: 1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 1 is present for follower follower778 02:32:20,114 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:20,115 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Log size is now 1 02:32:20,115 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:20,115 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 0, nextIndex: 1 02:32:20,115 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:20,115 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 1 is present for follower follower779 02:32:20,115 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:20,115 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 02:32:20,116 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower779: awaiting more completions to resolve 1 deferred callback(s) 02:32:20,123 PM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 02:32:20,123 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 9.409 ms 02:32:20,123 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 6.620 μs 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 0 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to 0 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader777: Applying state for log index 0 data zero 02:32:20,123 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader777: applyState called: zero 02:32:20,124 PM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 02:32:20,124 PM [follower779-writer-169] [DEBUG] EntryStoreCompleter - follower779: 1 completion(s) pending 02:32:20,124 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 9.981 ms 02:32:20,124 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:20,124 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: persisted 1 entries in 10.03 ms 02:32:20,124 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting for more work 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower779: running 1 completion(s) 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@487ff192 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower779: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@649c8b94 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Number of entries to be appended = 1 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Append entry to log one 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Append entry to log one 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower779: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:20,125 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 entries in 443.6 μs 02:32:20,125 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 480.9 μs 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Log size is now 2 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower779, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Log size is now 2 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached -1 (of expected -1) 02:32:20,125 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower779: awaiting more completions to resolve 1 deferred callback(s) 02:32:20,125 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 02:32:20,125 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower779 updated: matchIndex: 1, nextIndex: 2 02:32:20,126 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:20,126 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 1, nextIndex: 2 02:32:20,126 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:20,127 PM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 3.731 ms 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 12.21 μs 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 1 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 0 to 1 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 176.1 μs 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 1 data one 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: one 02:32:20,127 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - leader777: fakeSnapshot purging log to 0 for term 6 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 265.0 μs 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 83.48 μs 02:32:20,127 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:20,129 PM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 02:32:20,129 PM [follower779-writer-169] [DEBUG] EntryStoreCompleter - follower779: 1 completion(s) pending 02:32:20,129 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 3.711 ms 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@7eeaa004 02:32:20,129 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:20,129 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: persisted 1 entries in 3.787 ms 02:32:20,129 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting for more work 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower779: running 1 completion(s) 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Commit index set to 0 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - follower779: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 0 (of expected -1) 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - follower779: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@36f01ace 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 0 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from -1 to 0 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower778: Applying state for log index 0 data zero 02:32:20,129 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower778: applyState called: zero 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Commit index set to 0 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 0 (of expected -1) 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 0 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from -1 to 0 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower779: Applying state for log index 0 data zero 02:32:20,129 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower779: applyState called: zero 02:32:20,129 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 691.0 μs 02:32:20,129 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 entries in 632.9 μs 02:32:20,129 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): Commit index set to 1 02:32:20,129 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 02:32:20,129 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:20,129 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 1 02:32:20,129 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 0 to 1 02:32:20,129 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower778: Applying state for log index 1 data one 02:32:20,130 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower778: applyState called: one 02:32:20,129 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 120.7 μs 02:32:20,130 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:20,130 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower778: fakeSnapshot purging log to 0 for term 6 02:32:20,130 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: persisted 1 entries in 166.0 μs 02:32:20,130 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting for more work 02:32:20,130 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 92.73 μs 02:32:20,130 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 78.86 μs 02:32:20,130 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work [WARN] [06/27/2025 14:32:20.130] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower778] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower778, initialSyncDone=true] 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): Commit index set to 1 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - follower779 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower779 (Follower): Setting last applied to 1 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - follower779: Moving last applied index from 0 to 1 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower779: Applying state for log index 1 data one 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - follower779: applyState called: one 02:32:20,130 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SnapshotManager - follower779: fakeSnapshot purging log to 0 for term 6 02:32:20,130 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 entries in 814.6 μs 02:32:20,131 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: persisted 1 entries in 92.89 μs 02:32:20,131 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: waiting for more work [WARN] [06/27/2025 14:32:20.131] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower779] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower779, initialSyncDone=true] 02:32:20,317 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 02:32:20,317 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower779: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Persist data index=2 term=6 command=two 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:20,368 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 240.4 ms 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower778 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Append entry to log two 02:32:20,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:20,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Log size is now 2 02:32:20,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:20,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 1 (of expected -1) 02:32:20,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 02:32:20,369 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 2, nextIndex: 3 02:32:20,369 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 238.8 ms 02:32:20,369 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:20,370 PM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 02:32:20,370 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 2.244 ms 02:32:20,370 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (Leader): Setting last applied to 2 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 1 to 2 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 2 data two 02:32:20,370 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: two 02:32:20,371 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - leader777: fakeSnapshot purging log to 1 for term 6 02:32:20,371 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 468.5 μs 02:32:20,371 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 135.8 μs 02:32:20,371 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:20,371 PM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 02:32:20,371 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 02:32:20,372 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 02:32:20,371 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 2.750 ms 02:32:20,372 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:20,372 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@4552dcf6 02:32:20,437 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 02:32:20,437 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}]} 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower779: AppendEntries{term=6, leaderId=leader777, prevLogIndex=1, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=false}]} 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Commit index set to 2 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 2 (of expected -1) 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 2 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 1 to 2 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower778: Applying state for log index 2 data two 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower778: applyState called: two 02:32:20,557 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower778: fakeSnapshot purging log to 1 for term 6 02:32:20,557 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 185.8 ms 02:32:20,558 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 177.5 μs 02:32:20,558 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:20,569 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader777: Persist data index=3 term=6 command=three 02:32:20,569 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:20,569 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 198.4 ms 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 2 is present for follower follower779 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): sendAppendEntries: 3 is present for follower follower778 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (Leader): Sending AppendEntries to follower follower778: AppendEntries{term=6, leaderId=leader777, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]} 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader777, prevLogIndex=2, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}]} 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Append entry to log three 02:32:20,570 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:20,570 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 12.70 ms 02:32:20,571 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): Log size is now 2 02:32:20,571 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower778, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:20,571 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 2 (of expected -1) 02:32:20,571 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 02:32:20,571 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 3, nextIndex: 4 02:32:20,571 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:32:20,572 PM [leader777-writer-170] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 02:32:20,572 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 2.337 ms 02:32:20,572 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:20,572 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 02:32:20,572 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:20,572 PM [test-pekko.actor.default-dispatcher-11] [INFO] SnapshotManager - leader777: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=false] 02:32:20,573 PM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 02:32:20,573 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 2.476 ms 02:32:20,573 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:20,573 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 02:32:20,573 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 02:32:20,573 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@584c63e5 02:32:20,573 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader777: terminating thread VirtualThread[#907,leader777-writer-170]/waiting 02:32:20,574 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: received 1 entries in 1.756 ms 02:32:20,574 PM [test-pekko.actor.default-dispatcher-16] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) [ERROR] [06/27/2025 14:32:20.573] [test-pekko.actor.internal-dispatcher-10] [pekko://test/user/leader777] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:32:20,574 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 37.65 μs 02:32:20,574 PM [leader777-writer-170] [DEBUG] JournalWriteTask - leader777: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@cb7b73c stopped 02:32:20,574 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader777: journal closed 02:32:20,574 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader777: journal closed 02:32:20,574 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 02:32:20,574 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 02:32:20,576 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - leader777: Starting recovery with journal batch size 1000 02:32:20,576 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: started executor 02:32:20,576 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: started executor [WARN] [06/27/2025 14:32:20.576] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader777]: org.opendaylight.controller.cluster.common.actor.Monitor@7b20619c 02:32:20,576 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader777: metadata journal open: replayFrom=1 applyTo=2 02:32:20,577 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader777: journal open: firstIndex=1 lastIndex=4 replayFrom=1 fileEntries=0 02:32:20,577 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader777: journal open: applyTo=2 02:32:20,578 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@4dee8be2 started 02:32:20,578 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:20,579 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=5, votedFor=leader777 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader777: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader777: restoring local TermInfo{term=6, votedFor=leader777} 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=6, votedFor=leader777 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@70316035 02:32:20,580 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to -1 02:32:20,581 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 02:32:20,581 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=1, term=6, command=one] 02:32:20,581 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=2, term=6, command=two] 02:32:20,581 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: recovered journal DefaultLogEntry[index=3, term=6, command=three] 02:32:20,582 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader777: journal recovery completedin 1.693 ms with journalIndex=1 02:32:20,671 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:20,671 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:20,671 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:20,671 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:20,772 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:20,772 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:20,772 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:20,772 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:20,873 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:20,873 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:20,873 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:20,873 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:20,973 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:20,973 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:20,973 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:20,973 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,26 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader777 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:21,26 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader777 (Follower): Received ElectionTimeout - switching to Candidate 02:32:21,26 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader777 (Candidate): Election: Candidate has following voting peers: [follower779, follower778] 02:32:21,32 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader777: Set currentTerm=7, votedFor=leader777 02:32:21,32 PM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader777 (Candidate): Starting new election term 7 02:32:21,32 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} to peer follower779 02:32:21,32 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader777 (Candidate): Sending RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} to peer follower778 02:32:21,32 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader777 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 02:32:21,32 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower779 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:21,32 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower779 (Follower): Cluster state: [] 02:32:21,32 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower779 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} with higher term appears reachable, updating term. 02:32:21,32 PM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower779 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6}" message is greater than follower's term 6 - updating term 02:32:21,32 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:21,32 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Cluster state: [] 02:32:21,32 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} with higher term appears reachable, updating term. 02:32:21,32 PM [test-pekko.actor.default-dispatcher-11] [INFO] Follower - follower778 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6}" message is greater than follower's term 6 - updating term 02:32:21,45 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=7, votedFor=null 02:32:21,45 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower779 (Follower): In requestVote: RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 6 02:32:21,55 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=7, votedFor=null 02:32:21,55 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): In requestVote: RequestVote{term=7, candidateId=leader777, lastLogIndex=3, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 3, lastTerm: 6 02:32:21,60 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower779: Set currentTerm=7, votedFor=leader777 02:32:21,60 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower779 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader777 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader777 (Candidate): LastApplied index -1 is behind last index 3 - switching to PreLeader 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Election: Leader has following peers: [follower779, follower778] 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader777 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Persist data index=4 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: Append log entry and persist JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Replicate message: identifier: null, logIndex: 4, isSendImmediate: true 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 02:32:21,61 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 02:32:21,61 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 entries in 483.7 ms 02:32:21,67 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower778: Set currentTerm=7, votedFor=leader777 02:32:21,67 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - follower778 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 02:32:21,67 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 02:32:21,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 3, nextIndex: 4 02:32:21,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:21,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:21,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:21,67 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:21,67 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: -1 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): sendAppendEntries: 4 is present for follower follower778 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Sending AppendEntries to follower follower778: AppendEntries{term=7, leaderId=leader777, prevLogIndex=3, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader777, prevLogIndex=3, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:21,68 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader777 (PreLeader): sendAppendEntries: 4 is present for follower follower778 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): Number of entries to be appended = 1 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower778: Append log entry and persist JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): Log size is now 3 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower778, logLastIndex=4, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging -3 entries behind leader leader777 and reached 2 (of expected -1) 02:32:21,68 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower778: awaiting more completions to resolve 1 deferred callback(s) 02:32:21,68 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 495.1 ms 02:32:21,68 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader777 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower778 updated: matchIndex: 4, nextIndex: 5 02:32:21,68 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:21,68 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader777 (PreLeader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1 02:32:21,73 PM [leader777-writer-171] [DEBUG] EntryStoreCompleter - leader777: 1 completion(s) pending 02:32:21,73 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 11.64 ms 02:32:21,73 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader777: running 1 completion(s) 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader777: persist complete JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (PreLeader): Applying to log - commitIndex: 4, lastAppliedIndex: -1 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 0 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from -1 to 0 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 0 data zero 02:32:21,73 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: zero 02:32:21,74 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,74 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,74 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,74 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 1 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 0 to 1 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 1 data one 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: one 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 2 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 1 to 2 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 2 data two 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: two 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 3 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 2 to 3 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 3 data three 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader777: applyState called: three 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader777 (PreLeader): Setting last applied to 4 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader777: Moving last applied index from 3 to 4 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader777: Applying state for log index 4 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PreLeader - leader777 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 4, lastIndex: 4 02:32:21,74 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 entries in 1.083 ms 02:32:21,74 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Election: Leader has following peers: [follower779, follower778] 02:32:21,74 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 159.3 μs 02:32:21,74 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: waiting for more work 02:32:21,75 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower779: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1 02:32:21,75 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader777 (Leader): Checking sendAppendEntries for follower follower778: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1 02:32:21,75 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader777 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 7 02:32:21,77 PM [follower778-writer-168] [DEBUG] EntryStoreCompleter - follower778: 1 completion(s) pending 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower778: running 1 completion(s) 02:32:21,77 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 8.854 ms 02:32:21,77 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower778: persist complete JournaledLogEntry{index=4, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=false} 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower778: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@59e03e3a 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): Commit index set to 4 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower778 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 3 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 2 to 3 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower778: Applying state for log index 3 data three 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower778: applyState called: three 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower778 (Follower): Setting last applied to 4 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower778: Moving last applied index from 3 to 4 02:32:21,77 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower778: Applying state for log index 4 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:21,78 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 516.0 μs 02:32:21,78 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 117.6 μs 02:32:21,78 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: waiting for more work 02:32:21,174 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,174 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,174 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,174 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,187 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,275 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,275 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,275 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,275 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,306 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,375 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,375 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,375 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,375 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,417 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,475 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,476 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,476 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,476 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,536 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,576 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,576 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,576 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,576 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,656 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,676 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,676 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,676 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,676 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,766 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,777 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,777 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,777 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,777 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,877 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:21,887 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:21,977 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:21,978 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:21,978 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:21,978 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,6 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,78 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,126 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,178 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,178 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,179 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,179 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,247 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,279 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,357 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,379 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,385 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,396 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,398 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,477 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,502 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,502 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,502 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,502 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,596 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,602 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,603 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,603 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,603 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,703 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,703 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,703 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,703 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,716 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,804 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,804 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,804 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,804 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:22,905 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:22,905 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:22,905 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:22,905 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:22,956 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,5 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,5 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,5 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,6 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,76 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,106 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,106 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,106 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,106 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,197 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,207 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,207 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,207 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,207 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,308 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,308 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,308 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,308 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,316 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,408 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,408 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,408 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,408 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,437 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,509 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,509 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,509 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,509 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,556 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,609 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,609 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,609 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,609 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,676 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,710 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,710 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,710 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,710 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,797 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:23,810 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,811 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,811 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,811 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,911 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:23,911 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:23,911 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:23,911 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:23,917 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,12 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,12 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,12 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,12 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,37 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,113 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,113 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,113 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,113 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,157 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,213 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,213 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,213 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,213 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,276 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,314 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,314 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,314 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,314 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,397 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,415 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,415 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,415 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,415 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,515 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,515 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,515 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,515 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,517 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,615 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,615 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,615 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,615 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,637 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,716 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,716 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,716 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,716 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,746 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,817 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,817 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,817 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,817 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,867 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:24,917 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:24,917 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:24,917 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:24,917 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:24,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:25,18 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:25,18 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:25,18 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:25,18 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:25,106 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:25,119 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:25,119 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:25,119 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:25,119 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:25,219 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:25,219 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:25,219 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:25,219 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:25,227 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:25,320 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:25,320 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:25,320 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:25,320 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:25,346 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:25,421 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:25,421 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:25,421 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:25,421 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:25,457 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:25,521 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/TermInfo.properties 02:32:25,521 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/journal-v1-1.log 02:32:25,521 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest12255680085713055706/leader777/metadata-v1-1.log 02:32:25,521 PM [awaitility-thread] [DEBUG] RaftStorage - leader777: no eligible files found 02:32:25,566 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower778: Lagging 0 entries behind leader leader777 and reached 4 (of expected -1) 02:32:25,622 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower778-collector780#1839134829] 02:32:25,633 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower778] 02:32:25,634 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - follower778: terminating thread VirtualThread[#905,follower778-writer-168]/waiting 02:32:25,634 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: received 1 entries in 4.556 s 02:32:25,634 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: persisted 1 entries in 32.96 μs 02:32:25,634 PM [follower778-writer-168] [DEBUG] JournalWriteTask - follower778: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@148ed076 stopped 02:32:25,634 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower778: journal closed 02:32:25,634 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower778: journal closed 02:32:25,634 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower778: stopped executor with 0 remaining tasks 02:32:25,634 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower778: stopped executor with 0 remaining tasks 02:32:25,634 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower779-collector781#343101339] 02:32:25,635 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower779] 02:32:25,635 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - follower779: terminating thread VirtualThread[#906,follower779-writer-169]/waiting 02:32:25,635 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: received 1 entries in 5.504 s 02:32:25,635 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: persisted 1 entries in 16.96 μs 02:32:25,635 PM [follower779-writer-169] [DEBUG] JournalWriteTask - follower779: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1774b09a stopped 02:32:25,635 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower779: journal closed 02:32:25,635 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower779: journal closed 02:32:25,635 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks 02:32:25,635 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower779: stopped executor with 0 remaining tasks 02:32:25,635 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader777-collector782#-1049328365] 02:32:25,635 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader777] 02:32:25,636 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - leader777: terminating thread VirtualThread[#908,leader777-writer-171]/waiting 02:32:25,636 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: received 1 entries in 4.561 s 02:32:25,636 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: persisted 1 entries in 15.70 μs 02:32:25,636 PM [leader777-writer-171] [DEBUG] JournalWriteTask - leader777: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@4dee8be2 stopped 02:32:25,636 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader777: journal closed 02:32:25,636 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader777: journal closed 02:32:25,636 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 02:32:25,636 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader777: stopped executor with 0 remaining tasks 02:32:25,643 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderInstallsSnapshotWithRestartedFollowerDuringSnapshotInstallation starting [WARN] [06/27/2025 14:32:25.644] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower787]: org.opendaylight.controller.cluster.common.actor.Monitor@7ecfec99 02:32:25,645 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - follower787: Starting recovery with journal batch size 1000 02:32:25,645 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower787: started executor 02:32:25,646 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower787: started executor 02:32:25,646 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower787: metadata journal open: replayFrom=1 applyTo=0 02:32:25,647 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower787: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:25,647 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower787: journal open: applyTo=0 02:32:25,647 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@4d88a936 started 02:32:25,647 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work [WARN] [06/27/2025 14:32:25.648] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower788]: org.opendaylight.controller.cluster.common.actor.Monitor@42dceaf3 02:32:25,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/follower787/TermInfo.properties does not exist 02:32:25,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/follower787/journal-v1-1.log 02:32:25,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/follower787/metadata-v1-1.log 02:32:25,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: no eligible files found 02:32:25,648 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower787: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:25,650 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - follower788: Starting recovery with journal batch size 1000 02:32:25,650 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: started executor 02:32:25,650 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - follower788: started executor 02:32:25,650 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower788: metadata journal open: replayFrom=1 applyTo=0 02:32:25,651 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - follower788: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:25,651 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - follower788: journal open: applyTo=0 02:32:25,651 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@9ce919e started 02:32:25,651 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting for more work 02:32:25,652 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/follower788/TermInfo.properties does not exist 02:32:25,652 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/follower788/journal-v1-1.log 02:32:25,652 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower788: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/follower788/metadata-v1-1.log 02:32:25,652 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower788: no eligible files found 02:32:25,652 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower788: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:25,652 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=0, votedFor=null 02:32:25,652 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower787: Local TermInfo store seeded with TermInfo{term=0} 02:32:25,652 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower787: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@6c3facbc 02:32:25,652 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from -1 to -1 02:32:25,652 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower787: journal recovery completedin 184.1 μs with journalIndex=1 02:32:25,655 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=0, votedFor=null 02:32:25,655 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower788: Local TermInfo store seeded with TermInfo{term=0} 02:32:25,655 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower788: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@6537cfe2 02:32:25,655 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to -1 02:32:25,655 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - follower788: journal recovery completedin 142.5 μs with journalIndex=1 [WARN] [06/27/2025 14:32:25.750] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@7e8e9eb 02:32:25,751 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 02:32:25,751 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: started executor 02:32:25,751 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: started executor 02:32:25,752 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=0 02:32:25,752 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:25,752 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=0 02:32:25,752 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6158201a started 02:32:25,753 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:25,754 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties does not exist 02:32:25,754 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:25,754 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:25,754 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:25,754 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 02:32:25,755 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:25,758 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 02:32:25,758 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader786: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader786} 02:32:25,758 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@705f0a9c 02:32:25,758 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 02:32:25,758 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] JournalRecovery - leader786: journal recovery completedin 249.6 μs with journalIndex=1 02:32:26,176 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 02:32:26,176 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:26,176 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 02:32:26,176 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 02:32:26,180 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 02:32:26,180 PM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader786 (Candidate): Starting new election term 6 02:32:26,180 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} to peer follower788 02:32:26,180 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} to peer follower787 02:32:26,180 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower788 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:26,180 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower788 (Follower): Cluster state: [] 02:32:26,180 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower788 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:26,180 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower788 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:26,180 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 02:32:26,180 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:26,180 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 02:32:26,180 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:26,180 PM [test-pekko.actor.default-dispatcher-12] [INFO] Follower - follower787 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:26,186 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=6, votedFor=null 02:32:26,186 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:26,187 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=null 02:32:26,188 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower788 (Follower): In requestVote: RequestVote{term=6, candidateId=leader786, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower788: Set currentTerm=6, votedFor=leader786 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower788 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Election: Leader has following peers: [follower788, follower787] 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:26,192 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 02:32:26,192 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Last sync leader does not match current leader leader786, need to catch up to -1 02:32:26,193 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: -1, nextIndex: 0 02:32:26,193 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/27/2025 14:32:26.193] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=false] 02:32:26,194 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=6, votedFor=leader786 02:32:26,194 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:26,194 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Last sync leader does not match current leader leader786, need to catch up to -1 02:32:26,194 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: -1, nextIndex: 0 [WARN] [06/27/2025 14:32:26.194] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower787] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower787, initialSyncDone=false] 02:32:26,194 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:26,211 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=0 term=6 command=zero 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower788 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:26,211 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 458.5 ms 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): Append entry to log zero 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): Log size is now 1 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 02:32:26,211 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower788: awaiting more completions to resolve 1 deferred callback(s) 02:32:26,211 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 entries in 560.6 ms 02:32:26,211 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower787 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 0, nextIndex: 1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=1 term=6 command=one 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log zero 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 1 is present for follower follower788 02:32:26,212 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 564.9 ms 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower788: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 0 is present for follower follower787 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Log size is now 1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 02:32:26,212 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 0, nextIndex: 1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 02:32:26,212 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:26,212 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 1 is present for follower follower787 02:32:26,213 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:26,215 PM [follower788-writer-173] [DEBUG] EntryStoreCompleter - follower788: 1 completion(s) pending 02:32:26,215 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower788: running 1 completion(s) 02:32:26,215 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:26,215 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower788: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@3c5a5651 02:32:26,215 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: persisted 1 entries in 3.963 ms 02:32:26,215 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting for more work 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): Number of entries to be appended = 1 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): Append entry to log one 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower788: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): Log size is now 2 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower788 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower788, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:26,216 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 entries in 216.4 μs 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 02:32:26,216 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower788: awaiting more completions to resolve 1 deferred callback(s) 02:32:26,216 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower788 updated: matchIndex: 1, nextIndex: 2 02:32:26,216 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:26,217 PM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 02:32:26,217 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:26,217 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 5.336 ms 02:32:26,217 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 4.640 ms 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 02:32:26,217 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:26,217 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 5.770 μs 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 0 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to 0 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Applying state for log index 0 data zero 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@114f8b4a 02:32:26,217 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - leader786: applyState called: zero 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=0, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Append entry to log one 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:26,217 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 366.8 μs 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Log size is now 2 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached -1 (of expected -1) 02:32:26,217 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 02:32:26,217 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 1, nextIndex: 2 02:32:26,217 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:26,218 PM [follower788-writer-173] [DEBUG] EntryStoreCompleter - follower788: 1 completion(s) pending 02:32:26,218 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: persisted 1 entries in 2.229 ms 02:32:26,218 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting for more work 02:32:26,218 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower788: running 1 completion(s) 02:32:26,218 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower788: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:26,218 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower788: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@58d47f1e [WARN] [06/27/2025 14:32:26.218] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower788, initialSyncDone=true] 02:32:26,219 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:26,219 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 1.996 ms 02:32:26,219 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:26,219 PM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 02:32:26,219 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 2.429 ms 02:32:26,219 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 2.830 μs 02:32:26,219 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 69.97 μs 02:32:26,219 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@2da7ca3e 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Commit index set to 0 02:32:26,219 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 0 (of expected -1) 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 0 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from -1 to 0 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower787: Applying state for log index 0 data zero 02:32:26,219 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower787: applyState called: zero 02:32:26,219 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:26,219 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 483.1 μs 02:32:26,220 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:32:26,220 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 1 02:32:26,220 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 0 to 1 02:32:26,220 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 48.48 μs 02:32:26,220 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:26,220 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 1 data one 02:32:26,220 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: one [WARN] [06/27/2025 14:32:26.220] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower787] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower787, initialSyncDone=true] 02:32:26,220 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader786: fakeSnapshot purging log to 0 for term 6 02:32:26,220 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 719.9 μs 02:32:26,220 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 44.68 μs 02:32:26,220 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): Commit index set to 1 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 1 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 0 to 1 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower787: Applying state for log index 1 data one 02:32:26,220 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower787: applyState called: one 02:32:26,221 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 942.0 μs 02:32:26,221 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 42.13 μs 02:32:26,221 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:26,221 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower787: fakeSnapshot purging log to 0 for term 6 02:32:26,426 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): Commit index set to 1 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower788 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 0 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from -1 to 0 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower788: Applying state for log index 0 data zero 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower788: applyState called: zero 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower788 (Follower): Setting last applied to 1 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower788: Moving last applied index from 0 to 1 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower788: Applying state for log index 1 data one 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower788: applyState called: one 02:32:26,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower788: fakeSnapshot purging log to 0 for term 6 02:32:26,427 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 entries in 209.2 ms 02:32:26,427 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: persisted 1 entries in 173.0 μs 02:32:26,427 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: waiting for more work 02:32:26,547 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower788: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,547 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,563 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower788: terminating thread VirtualThread[#911,follower788-writer-173]/waiting 02:32:26,563 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: received 1 entries in 136.1 ms 02:32:26,564 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: persisted 1 entries in 42.43 μs 02:32:26,564 PM [follower788-writer-173] [DEBUG] JournalWriteTask - follower788: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@9ce919e stopped 02:32:26,564 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower788: journal closed 02:32:26,564 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower788: journal closed 02:32:26,564 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks 02:32:26,564 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower788: stopped executor with 0 remaining tasks 02:32:26,666 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) [WARN] [06/27/2025 14:32:26.667] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:26,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) [WARN] [06/27/2025 14:32:26.787] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [06/27/2025 14:32:26.907] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:26,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,968 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=2 term=6 command=two 02:32:26,968 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:26,969 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 748.6 ms 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=3 term=6 command=three 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 [WARN] [06/27/2025 14:32:26.969] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [06/27/2025 14:32:26.969] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 02:32:26,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=6, command=two, pending=true}]} 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=4 term=6 command=four 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 02:32:26,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true 02:32:26,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:32:26,969 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 02:32:26,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Append entry to log two 02:32:26,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=2, term=6, command=two, pending=true} [WARN] [06/27/2025 14:32:26.969] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 02:32:26,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Log size is now 2 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=5 term=6 command=five 02:32:26,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=2, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 02:32:26,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true 02:32:26,970 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 02:32:26,970 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 749.1 ms 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 2, leaderLastIndex: 5, leaderSnapShotIndex: 0 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 2 is present for follower follower787 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 2, nextIndex: 3 [WARN] [06/27/2025 14:32:26.970] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 3, leaderLastIndex: 5, leaderSnapShotIndex: 0 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 3 is present for follower follower787 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}]} 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader786: Persist data index=6 term=6 command=six 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Replicate message: identifier: MockIdentifier{value=six}, logIndex: 6, isSendImmediate: true 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: 2, leaderLastIndex: 6, leaderSnapShotIndex: 0 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 0 02:32:26,970 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 3 is present for follower follower787 [WARN] [06/27/2025 14:32:26.970] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:26,981 PM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 02:32:26,981 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 12.08 ms 02:32:26,981 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 4 entries in 7.710 μs 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=true} 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader786 (Leader): Setting last applied to 2 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 1 to 2 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Applying state for log index 2 data two 02:32:26,981 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader786: applyState called: two 02:32:26,982 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader786: fakeSnapshot purging log to 1 for term 6 02:32:27,13 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:27,13 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:27,13 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 43.44 ms 02:32:27,13 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:27,13 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=2, term=6, command=two, pending=false} 02:32:27,13 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@618aa2c3 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=2, prevLogTerm=6, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=6, command=three, pending=true}, JournaledLogEntry{index=4, term=6, command=four, pending=true}, JournaledLogEntry{index=5, term=6, command=five, pending=true}]} 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 3 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Append entry to log three 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Append entry to log four 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=4, term=6, command=four, pending=true} 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Append entry to log five 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=5, term=6, command=five, pending=true} 02:32:27,14 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 555.4 μs 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Log size is now 5 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=5, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 1 (of expected -1) 02:32:27,14 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 3 deferred callback(s) 02:32:27,14 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 5, nextIndex: 6 02:32:27,14 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 1 02:32:27,14 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): sendAppendEntries: 6 is present for follower follower787 02:32:27,14 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Sending AppendEntries to follower follower787: AppendEntries{term=6, leaderId=leader786, prevLogIndex=5, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]} [WARN] [06/27/2025 14:32:27.027] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=6, leaderId=leader786, prevLogIndex=1, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 02:32:27,50 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:27,50 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:27,50 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=false} 02:32:27,50 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@39e49fef 02:32:27,50 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 2 deferred callback(s) 02:32:27,50 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 35.83 ms 02:32:27,50 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 2 entries in 15.60 μs 02:32:27,69 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,69 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,70 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,70 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,86 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 2 completion(s) pending 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: running 2 completion(s) 02:32:27,86 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 2 entries in 36.35 ms 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=4, term=6, command=four, pending=false} 02:32:27,86 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@63ee81bc 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=5, term=6, command=five, pending=false} 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@43dd6ccc 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader786, prevLogIndex=5, prevLogTerm=6, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=6, term=6, command=six, pending=true}]} 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Append entry to log six 02:32:27,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=6, term=6, command=six, pending=true} 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Log size is now 6 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): Commit index set to 2 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower787, logLastIndex=6, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:27,87 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 389.5 μs 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 2 (of expected -1) 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 6 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 2 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 1 to 2 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower787: Applying state for log index 2 data two 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower787: applyState called: two 02:32:27,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 6, nextIndex: 7 02:32:27,87 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 1 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower787: fakeSnapshot purging log to 1 for term 6 02:32:27,87 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 02:32:27,92 PM [leader786-writer-174] [DEBUG] EntryStoreCompleter - leader786: 4 completion(s) pending 02:32:27,92 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 4 entries in 110.7 ms 02:32:27,92 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - leader786: running 4 completion(s) 02:32:27,92 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 11.08 μs 02:32:27,92 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=3, term=6, command=three, pending=true} 02:32:27,92 PM [test-pekko.actor.default-dispatcher-12] [INFO] SnapshotManager - leader786: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=6, lastIndex=3, lastTerm=6, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=4, mandatoryTrim=false] 02:32:27,92 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 149.9 μs 02:32:27,92 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:27,92 PM [test-pekko.actor.default-dispatcher-16] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [ERROR] [06/27/2025 14:32:27.092] [test-pekko.actor.internal-dispatcher-2] [pekko://test/user/leader786] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:32:27,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#912,leader786-writer-174]/waiting 02:32:27,93 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: received 1 entries in 1.278 ms 02:32:27,93 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 32.10 μs 02:32:27,93 PM [leader786-writer-174] [DEBUG] JournalWriteTask - leader786: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6158201a stopped 02:32:27,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader786: journal closed 02:32:27,93 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader786: journal closed 02:32:27,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 02:32:27,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks [WARN] [06/27/2025 14:32:27.094] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@3cd360a6 02:32:27,96 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:27,96 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 8.916 ms 02:32:27,96 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 3.790 μs 02:32:27,96 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:27,96 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=6, term=6, command=six, pending=false} 02:32:27,96 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@36bed037 02:32:27,96 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 128.8 μs 02:32:27,96 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:27,96 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 02:32:27,96 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: started executor 02:32:27,96 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader786: started executor 02:32:27,97 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=2 02:32:27,98 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=7 replayFrom=1 fileEntries=0 02:32:27,98 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=2 02:32:27,98 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@70d62402 started 02:32:27,98 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:27,100 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 02:32:27,100 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,100 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,100 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,100 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,100 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader786: restoring local TermInfo{term=6, votedFor=leader786} 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=6, votedFor=leader786 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@70484874 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=1, term=6, command=one] 02:32:27,101 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=2, term=6, command=two] 02:32:27,102 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=3, term=6, command=three] 02:32:27,102 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=4, term=6, command=four] 02:32:27,102 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=5, term=6, command=five] 02:32:27,102 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=6, term=6, command=six] 02:32:27,102 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] JournalRecovery - leader786: journal recovery completedin 1.511 ms with journalIndex=1 02:32:27,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,170 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,271 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,372 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,372 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,372 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,372 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,472 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,546 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:27,546 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 02:32:27,546 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 02:32:27,550 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 02:32:27,550 PM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader786 (Candidate): Starting new election term 7 02:32:27,550 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} to peer follower788 02:32:27,550 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} to peer follower787 02:32:27,550 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 7 02:32:27,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:27,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 02:32:27,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} with higher term appears reachable, updating term. 02:32:27,551 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower787 (Follower): Term 7 in "RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6}" message is greater than follower's term 6 - updating term [WARN] [06/27/2025 14:32:27.551] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} 02:32:27,554 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=7, votedFor=null 02:32:27,555 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=7, candidateId=leader786, lastLogIndex=6, lastLogTerm=6} - currentTerm: 7, votedFor: null, lastIndex: 6, lastTerm: 6 02:32:27,558 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=7, votedFor=leader786 02:32:27,558 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=7, voteGranted=true} 02:32:27,558 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=7, voteGranted=true}, my term is 7 02:32:27,558 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=7, voteGranted=true}, current voteCount: 1 02:32:27,558 PM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader786 (Candidate): LastApplied index -1 is behind last index 6 - switching to PreLeader 02:32:27,558 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Election: Leader has following peers: [follower788, follower787] 02:32:27,558 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 [WARN] [06/27/2025 14:32:27.559] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=7, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 6, leaderSnapShotIndex: -1 02:32:27,559 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 6, nextIndex: 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 02:32:27,559 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: -1 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader786: Persist data index=7 term=7 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Replicate message: identifier: null, logIndex: 7, isSendImmediate: true 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:27,560 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: received 1 entries in 461.7 ms 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 7 is present for follower follower787 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader786 (PreLeader): Sending AppendEntries to follower follower787: AppendEntries{term=7, leaderId=leader786, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} [WARN] [06/27/2025 14:32:27.560] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=7, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=7, leaderId=leader786, prevLogIndex=6, prevLogTerm=6, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 02:32:27,560 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:27,561 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Log size is now 6 02:32:27,561 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=7, success=true, followerId=follower787, logLastIndex=7, logLastTerm=7, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:27,561 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 02:32:27,561 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 02:32:27,561 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 465.0 ms 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 7, nextIndex: 8 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 7 02:32:27,561 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:27,562 PM [leader786-writer-175] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 02:32:27,562 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 2.089 ms 02:32:27,562 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:27,562 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 02:32:27,562 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:27,563 PM [test-pekko.actor.default-dispatcher-16] [INFO] SnapshotManager - leader786: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=7, lastTerm=7, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=8, mandatoryTrim=false] 02:32:27,563 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:27,563 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 2.685 ms 02:32:27,563 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:27,564 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:27,564 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=false} 02:32:27,564 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@2d9f710a [ERROR] [06/27/2025 14:32:27.563] [test-pekko.actor.internal-dispatcher-2] [pekko://test/user/leader786] null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:32:27,564 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#914,leader786-writer-175]/waiting 02:32:27,564 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: received 1 entries in 1.869 ms 02:32:27,564 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 27.07 μs 02:32:27,564 PM [leader786-writer-175] [DEBUG] JournalWriteTask - leader786: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@70d62402 stopped 02:32:27,564 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: journal closed 02:32:27,564 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader786: journal closed 02:32:27,564 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 02:32:27,565 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 02:32:27,565 PM [test-pekko.actor.default-dispatcher-6] [ERROR] OneForOneStrategy - null java.lang.UnsupportedOperationException at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRaftStorage.lastSequenceNumber(PekkoRaftStorage.java:141) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:342) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.SnapshotManager.capture(SnapshotManager.java:337) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.captureSnapshotIfReady(ReplicatedLogImpl.java:93) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.lambda$submitCommand$0(RaftActor.java:747) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$2(ReplicatedLogImpl.java:147) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.invokeSync(ReplicatedLogImpl.java:160) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl.lambda$appendSubmitted$3(ReplicatedLogImpl.java:145) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:37) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$UncheckedPersistCallback.invoke(ReplicatedLogImpl.java:32) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.JournalWriteTask.lambda$runBatch$0(JournalWriteTask.java:308) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.runCompletions(EntryStoreCompleter.java:167) at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.spi.EntryStoreCompleter.completeWhilePending(EntryStoreCompleter.java:135) at org.opendaylight.controller.cluster.commons@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage.run(ExecuteInSelfMessage.java:25) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.library@2.13.16/scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) [WARN] [06/27/2025 14:32:27.565] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader786]: org.opendaylight.controller.cluster.common.actor.Monitor@40483538 02:32:27,566 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - leader786: Starting recovery with journal batch size 1000 02:32:27,567 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: started executor 02:32:27,567 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader786: started executor 02:32:27,567 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: metadata journal open: replayFrom=1 applyTo=2 02:32:27,569 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader786: journal open: firstIndex=1 lastIndex=8 replayFrom=1 fileEntries=0 02:32:27,569 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader786: journal open: applyTo=2 02:32:27,569 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@22f58e1c started 02:32:27,569 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:27,571 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 02:32:27,571 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,571 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,571 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,571 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,571 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=5, votedFor=leader786 02:32:27,572 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader786: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:27,572 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - leader786: restoring local TermInfo{term=7, votedFor=leader786} 02:32:27,572 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=7, votedFor=leader786 02:32:27,572 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader786: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7a44ff1b 02:32:27,572 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to -1 02:32:27,572 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=0, term=6, command=zero] 02:32:27,573 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=1, term=6, command=one] 02:32:27,573 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=2, term=6, command=two] 02:32:27,573 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,573 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,573 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,573 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,573 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=3, term=6, command=three] 02:32:27,573 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=4, term=6, command=four] 02:32:27,573 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=5, term=6, command=five] 02:32:27,574 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=6, term=6, command=six] 02:32:27,574 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: recovered journal DefaultLogEntry[index=7, term=7, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7] 02:32:27,574 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - leader786: journal recovery completedin 2.404 ms with journalIndex=1 02:32:27,674 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,674 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,674 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,674 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,774 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,774 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,774 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,774 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,875 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,875 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,875 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,875 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:27,975 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:27,975 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:27,975 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:27,975 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:28,76 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,76 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,76 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,76 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:28,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader786 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:28,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader786 (Follower): Received ElectionTimeout - switching to Candidate 02:32:28,77 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): Election: Candidate has following voting peers: [follower788, follower787] 02:32:28,81 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader786: Set currentTerm=8, votedFor=leader786 02:32:28,81 PM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader786 (Candidate): Starting new election term 8 02:32:28,81 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} to peer follower788 02:32:28,82 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader786 (Candidate): Sending RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} to peer follower787 02:32:28,82 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader786 (Follower) :- Switching from behavior Follower to Candidate, election term: 8 [WARN] [06/27/2025 14:32:28.082] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} 02:32:28,82 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:28,82 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Cluster state: [] 02:32:28,82 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): Candidate in requestVote:RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} with higher term appears reachable, updating term. 02:32:28,82 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower787 (Follower): Term 8 in "RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7}" message is greater than follower's term 7 - updating term 02:32:28,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=8, votedFor=null 02:32:28,86 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): In requestVote: RequestVote{term=8, candidateId=leader786, lastLogIndex=7, lastLogTerm=7} - currentTerm: 8, votedFor: null, lastIndex: 7, lastTerm: 7 02:32:28,88 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower787: Set currentTerm=8, votedFor=leader786 02:32:28,88 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower787 (Follower): requestVote returning: RequestVoteReply{term=8, voteGranted=true} 02:32:28,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader786 (Candidate): RaftRPC message received RequestVoteReply{term=8, voteGranted=true}, my term is 8 02:32:28,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader786 (Candidate): handleRequestVoteReply: RequestVoteReply{term=8, voteGranted=true}, current voteCount: 1 02:32:28,88 PM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader786 (Candidate): LastApplied index -1 is behind last index 7 - switching to PreLeader 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Election: Leader has following peers: [follower788, follower787] 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 7, leaderSnapShotIndex: -1 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader786 (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Persist data index=8 term=8 command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:28,89 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader786: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Replicate message: identifier: null, logIndex: 8, isSendImmediate: true 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 [WARN] [06/27/2025 14:32:28.089] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:28,89 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 entries in 519.9 ms 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 7, nextIndex: 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 8 is present for follower follower787 02:32:28,89 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Sending AppendEntries to follower follower787: AppendEntries{term=8, leaderId=leader786, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} [WARN] [06/27/2025 14:32:28.089] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower787 (Follower): handleAppendEntries: AppendEntries{term=8, leaderId=leader786, prevLogIndex=7, prevLogTerm=7, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true}]} 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower787 (Follower): Number of entries to be appended = 1 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower787 (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower787 (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower787: Append log entry and persist JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower787 (Follower): Log size is now 7 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower787 (Follower): handleAppendEntries returning : AppendEntriesReply{term=8, success=true, followerId=follower787, logLastIndex=8, logLastTerm=8, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower787: Lagging -3 entries behind leader leader786 and reached 2 (of expected -1) 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:28,90 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 526.3 ms 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower787: awaiting more completions to resolve 1 deferred callback(s) 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): sendAppendEntries: 8 is present for follower follower787 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): handleAppendEntriesReply - FollowerLogInformation for follower787 updated: matchIndex: 8, nextIndex: 9 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:28,90 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader786 (PreLeader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:28,91 PM [follower787-writer-172] [DEBUG] EntryStoreCompleter - follower787: 1 completion(s) pending 02:32:28,91 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower787: running 1 completion(s) 02:32:28,91 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 1.518 ms 02:32:28,91 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:28,91 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower787: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=false} 02:32:28,91 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - follower787: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@2303a236 02:32:28,91 PM [leader786-writer-176] [DEBUG] EntryStoreCompleter - leader786: 1 completion(s) pending 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryStoreCompleter - leader786: running 1 completion(s) 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader786: persist complete JournaledLogEntry{index=8, term=8, command=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7, pending=true} 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 6 does not match the current term 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 6 does not match the current term 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 6 does not match the current term 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 3 - retrieved log entry with index 3, term 6 does not match the current term 8 02:32:28,92 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 2.396 ms 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 4 - retrieved log entry with index 4, term 6 does not match the current term 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 5 - retrieved log entry with index 5, term 6 does not match the current term 8 02:32:28,92 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 6 - retrieved log entry with index 6, term 6 does not match the current term 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Not updating commit index to 7 - retrieved log entry with index 7, term 7 does not match the current term 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (PreLeader): Applying to log - commitIndex: 8, lastAppliedIndex: -1 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 0 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from -1 to 0 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 0 data zero 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: zero 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 1 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 0 to 1 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 1 data one 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: one 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 2 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 1 to 2 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 2 data two 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: two 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 3 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 2 to 3 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 3 data three 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: three 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 4 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 3 to 4 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 4 data four 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: four 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 5 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 4 to 5 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 5 data five 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: five 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 6 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 5 to 6 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 6 data six 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader786: applyState called: six 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 7 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 6 to 7 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader786 (PreLeader): Setting last applied to 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader786: Moving last applied index from 7 to 8 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader786: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PreLeader - leader786 (PreLeader): Received ApplyState[identifier=null, entry=JournaledLogEntry{index=0, term=6, command=zero, pending=false}] - lastApplied: 8, lastIndex: 8 02:32:28,92 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 entries in 495.3 μs 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): Election: Leader has following peers: [follower788, follower787] 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower788: active: false, followerNextIndex: -1, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:28,92 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 194.7 μs 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader786 (Leader): Checking sendAppendEntries for follower follower787: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: -1 02:32:28,92 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: waiting for more work 02:32:28,92 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader786 (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 8 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): Commit index set to 8 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower787 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 2, lastIndex(): 8 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 3 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 2 to 3 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 3 data three 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: three [WARN] [06/27/2025 14:32:28.093] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 4 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 3 to 4 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 4 data four 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: four 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 5 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 4 to 5 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 5 data five 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: five 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 6 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 5 to 6 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 6 data six 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower787: applyState called: six 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 7 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 6 to 7 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 7 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower787 (Follower): Setting last applied to 8 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower787: Moving last applied index from 7 to 8 02:32:28,93 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower787: Applying state for log index 8 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@d03c8e7 02:32:28,93 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 1.843 ms 02:32:28,94 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 221.7 μs 02:32:28,94 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: waiting for more work 02:32:28,177 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,177 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,177 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,177 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:28.207] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,207 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:28,277 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,277 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,277 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,277 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:28.327] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,327 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:28,378 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,378 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,378 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,378 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:28.447] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,447 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:28,479 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,479 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,479 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,479 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:28.566] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,566 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:28,580 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,580 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,580 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,580 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:28,680 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,680 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,680 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,680 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:28,687 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:28.687] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,781 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,781 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,781 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,781 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:28,807 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:28.807] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,882 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,882 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,882 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,882 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:28.927] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:28,927 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:28,983 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:28,983 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:28,983 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:28,983 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:29.047] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,47 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:29,83 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,84 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,84 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,84 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:29,166 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:29.166] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,184 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,184 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,184 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,184 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:29,285 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,285 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,285 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,285 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:29.286] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,287 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:29,386 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,386 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,386 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,386 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:29,409 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:29.409] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,487 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,488 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,488 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,488 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:29,526 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:29.526] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,589 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,589 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,589 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,589 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:29.646] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,646 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:29,689 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,689 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,690 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,690 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:29,767 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:29.767] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,790 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,790 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,790 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,790 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:29.887] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:29,887 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:29,891 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,891 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,891 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,891 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:29,991 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:29,991 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:29,991 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:29,992 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:30,6 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:30.006] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,92 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,92 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,92 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,92 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.127] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,127 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,193 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,193 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,193 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,193 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.247] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,247 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,293 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,293 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,293 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,293 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.367] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,367 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,394 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,394 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,394 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,394 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.487] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,487 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,494 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,494 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,494 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,494 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:30,595 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,595 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,595 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,595 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.607] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,607 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,695 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,695 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,695 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,695 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.726] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,726 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,796 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,796 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,796 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,796 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:30,847 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:30.847] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,896 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,896 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,896 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,896 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:30.967] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:30,967 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:30,997 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:30,997 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:30,997 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:30,997 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:31.087] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,87 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:31,97 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,97 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,97 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,97 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:31,198 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,198 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,198 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,198 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:31,206 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:31.206] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,298 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,298 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,299 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,299 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:31.327] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,327 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:31,399 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,399 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,399 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,399 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:31.447] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,447 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:31,500 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,500 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,500 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,500 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:31,566 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:31.566] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,600 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,600 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,600 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,600 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found [WARN] [06/27/2025 14:32:31.687] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,687 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) 02:32:31,701 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,701 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,701 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,701 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:31,802 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,802 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,802 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,802 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:31,807 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:31.807] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:31,902 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/TermInfo.properties 02:32:31,902 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/journal-v1-1.log 02:32:31,902 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest7238721271196295653/leader786/metadata-v1-1.log 02:32:31,902 PM [awaitility-thread] [DEBUG] RaftStorage - leader786: no eligible files found 02:32:31,926 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower787: Lagging 0 entries behind leader leader786 and reached 8 (of expected -1) [WARN] [06/27/2025 14:32:31.926] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower788] received dead letter from TestActor[pekko://test/user/leader786]: AppendEntries{term=8, leaderId=leader786, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 02:32:32,4 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower787-collector789#-359261404] 02:32:32,15 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower787] 02:32:32,15 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower787: terminating thread VirtualThread[#910,follower787-writer-172]/waiting 02:32:32,15 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: received 1 entries in 3.921 s 02:32:32,15 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: persisted 1 entries in 30.16 μs 02:32:32,15 PM [follower787-writer-172] [DEBUG] JournalWriteTask - follower787: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@4d88a936 stopped 02:32:32,15 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower787: journal closed 02:32:32,15 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower787: journal closed 02:32:32,15 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: stopped executor with 0 remaining tasks 02:32:32,15 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower787: stopped executor with 0 remaining tasks 02:32:32,16 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower788-collector790#-1744399601] 02:32:32,16 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower788] 02:32:32,16 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader786-collector791#191327014] 02:32:32,16 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader786] 02:32:32,16 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader786: terminating thread VirtualThread[#915,leader786-writer-176]/waiting 02:32:32,17 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: received 1 entries in 3.924 s 02:32:32,17 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: persisted 1 entries in 18.44 μs 02:32:32,17 PM [leader786-writer-176] [DEBUG] JournalWriteTask - leader786: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@22f58e1c stopped 02:32:32,17 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader786: journal closed 02:32:32,17 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader786: journal closed 02:32:32,17 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 02:32:32,17 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader786: stopped executor with 0 remaining tasks 02:32:32,20 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - testLeaderSnapshotTriggeredByMemoryThresholdExceededWithLaggingFollower starting [WARN] [06/27/2025 14:32:32.021] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower796]: org.opendaylight.controller.cluster.common.actor.Monitor@2db9e07 02:32:32,22 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - follower796: Starting recovery with journal batch size 1000 02:32:32,22 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower796: started executor 02:32:32,22 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower796: started executor 02:32:32,23 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower796: metadata journal open: replayFrom=1 applyTo=0 02:32:32,23 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower796: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:32,23 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower796: journal open: applyTo=0 02:32:32,23 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@7056f707 started 02:32:32,24 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting for more work 02:32:32,25 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/follower796/TermInfo.properties does not exist 02:32:32,25 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/follower796/journal-v1-1.log 02:32:32,25 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/follower796/metadata-v1-1.log 02:32:32,25 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: no eligible files found 02:32:32,25 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower796: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [06/27/2025 14:32:32.025] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower797]: org.opendaylight.controller.cluster.common.actor.Monitor@22451554 02:32:32,26 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower797: Starting recovery with journal batch size 1000 02:32:32,26 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower797: started executor 02:32:32,26 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower797: started executor 02:32:32,27 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower797: metadata journal open: replayFrom=1 applyTo=0 02:32:32,27 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower797: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:32,27 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower797: journal open: applyTo=0 02:32:32,28 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@11278b04 started 02:32:32,28 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting for more work 02:32:32,28 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=0, votedFor=null 02:32:32,28 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower796: Local TermInfo store seeded with TermInfo{term=0} 02:32:32,28 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower796: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1ca141e1 02:32:32,28 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from -1 to -1 02:32:32,28 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - follower796: journal recovery completedin 242.0 μs with journalIndex=1 [WARN] [06/27/2025 14:32:32.029] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader795]: org.opendaylight.controller.cluster.common.actor.Monitor@70ce42 02:32:32,30 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActor - leader795: Starting recovery with journal batch size 1000 02:32:32,30 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/follower797/TermInfo.properties does not exist 02:32:32,30 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader795: started executor 02:32:32,30 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftStorage - leader795: started executor 02:32:32,30 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower797: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/follower797/journal-v1-1.log 02:32:32,30 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower797: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/follower797/metadata-v1-1.log 02:32:32,30 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower797: no eligible files found 02:32:32,30 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower797: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:32,30 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader795: metadata journal open: replayFrom=1 applyTo=0 02:32:32,31 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryJournalV1 - leader795: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 02:32:32,31 PM [test-pekko.actor.default-dispatcher-12] [INFO] PekkoRaftStorage - leader795: journal open: applyTo=0 02:32:32,31 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@573154e2 started 02:32:32,31 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting for more work 02:32:32,32 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/leader795/TermInfo.properties does not exist 02:32:32,32 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader795: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/leader795/journal-v1-1.log 02:32:32,32 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader795: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest16988578198107041506/leader795/metadata-v1-1.log 02:32:32,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader795: no eligible files found 02:32:32,33 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=5, votedFor=leader795 02:32:32,33 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader795: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:32:32,35 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=0, votedFor=null 02:32:32,35 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower797: Local TermInfo store seeded with TermInfo{term=0} 02:32:32,35 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower797: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@e02d188 02:32:32,35 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - follower797: Moving last applied index from -1 to -1 02:32:32,36 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] JournalRecovery - follower797: journal recovery completedin 264.1 μs with journalIndex=1 02:32:32,36 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=5, votedFor=leader795 02:32:32,36 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader795: Local TermInfo store seeded with TermInfo{term=5, votedFor=leader795} 02:32:32,36 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader795: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@608f2711 02:32:32,36 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from -1 to -1 02:32:32,36 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] JournalRecovery - leader795: journal recovery completedin 230.1 μs with journalIndex=1 02:32:32,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 02:32:32,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader795 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed) 02:32:32,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - leader795 (Follower): Received ElectionTimeout - switching to Candidate 02:32:32,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader795 (Candidate): Election: Candidate has following voting peers: [follower797, follower796] 02:32:32,540 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PropertiesTermInfoStore - leader795: Set currentTerm=6, votedFor=leader795 02:32:32,540 PM [test-pekko.actor.default-dispatcher-12] [INFO] Candidate - leader795 (Candidate): Starting new election term 6 02:32:32,540 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader795 (Candidate): Sending RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} to peer follower797 02:32:32,540 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Candidate - leader795 (Candidate): Sending RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} to peer follower796 02:32:32,540 PM [test-pekko.actor.default-dispatcher-12] [INFO] RaftActorBehavior - leader795 (Follower) :- Switching from behavior Follower to Candidate, election term: 6 02:32:32,540 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower797 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:32,540 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower797 (Follower): Cluster state: [] 02:32:32,540 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower797 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:32,540 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower797 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:32,542 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower796 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:32:32,542 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower796 (Follower): Cluster state: [] 02:32:32,542 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower796 (Follower): Candidate in requestVote:RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 02:32:32,542 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower796 (Follower): Term 6 in "RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 02:32:32,544 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=6, votedFor=null 02:32:32,545 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower797 (Follower): In requestVote: RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:32,547 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=6, votedFor=null 02:32:32,547 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower797: Set currentTerm=6, votedFor=leader795 02:32:32,547 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower797 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:32,547 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader795 (Candidate): RaftRPC message received RequestVoteReply{term=6, voteGranted=true}, my term is 6 02:32:32,547 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader795 (Candidate): handleRequestVoteReply: RequestVoteReply{term=6, voteGranted=true}, current voteCount: 1 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): Election: Leader has following peers: [follower797, follower796] 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader795 (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower797: Last sync leader does not match current leader leader795, need to catch up to -1 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower797 updated: matchIndex: -1, nextIndex: 0 02:32:32,548 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/27/2025 14:32:32.548] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower797] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower797, initialSyncDone=false] 02:32:32,547 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower796 (Follower): In requestVote: RequestVote{term=6, candidateId=leader795, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 02:32:32,551 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower796: Set currentTerm=6, votedFor=leader795 02:32:32,551 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower796 (Follower): requestVote returning: RequestVoteReply{term=6, voteGranted=true} 02:32:32,551 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Last sync leader does not match current leader leader795, need to catch up to -1 02:32:32,551 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: -1, nextIndex: 0 02:32:32,551 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [06/27/2025 14:32:32.551] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower796] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower796, initialSyncDone=false] 02:32:32,592 PM [main] [INFO] ReplicationAndSnapshotsWithLaggingFollowerIntegrationTest - Leader created and elected 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader795: Persist data index=0 term=6 command=zero 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader795: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 0 is present for follower follower797 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower797: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:32,592 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 entries in 561.4 ms 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 0 is present for follower follower796 02:32:32,592 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower796: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): Number of entries to be appended = 1 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): Append entry to log zero 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower797: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=6, command=zero, pending=true}]} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Number of entries to be appended = 1 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Append entry to log zero 02:32:32,593 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 entries in 565.3 ms 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower796: Append log entry and persist JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Log size is now 1 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower796, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached -1 (of expected -1) 02:32:32,593 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 entries in 569.6 ms 02:32:32,593 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: awaiting more completions to resolve 1 deferred callback(s) 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): Log size is now 1 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower797 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower797, logLastIndex=0, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached -1 (of expected -1) 02:32:32,593 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower797: awaiting more completions to resolve 1 deferred callback(s) 02:32:32,593 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: 0, nextIndex: 1 02:32:32,594 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:32,594 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower797 updated: matchIndex: 0, nextIndex: 1 02:32:32,594 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:32:32,599 PM [leader795-writer-179] [DEBUG] EntryStoreCompleter - leader795: 1 completion(s) pending 02:32:32,599 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: persisted 1 entries in 6.801 ms 02:32:32,599 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting for more work 02:32:32,599 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryStoreCompleter - leader795: running 1 completion(s) 02:32:32,599 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] ReplicatedLogImpl - leader795: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=true} 02:32:32,599 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader795 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:32:32,600 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorBehavior - leader795 (Leader): Setting last applied to 0 02:32:32,600 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from -1 to 0 02:32:32,600 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader795: Applying state for log index 0 data zero 02:32:32,600 PM [test-pekko.actor.default-dispatcher-11] [INFO] MockRaftActor - leader795: applyState called: zero 02:32:32,600 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 entries in 711.1 μs 02:32:32,600 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: persisted 1 entries in 130.9 μs 02:32:32,600 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting for more work 02:32:32,600 PM [follower796-writer-177] [DEBUG] EntryStoreCompleter - follower796: 1 completion(s) pending 02:32:32,600 PM [follower797-writer-178] [DEBUG] EntryStoreCompleter - follower797: 1 completion(s) pending 02:32:32,600 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: persisted 1 entries in 7.108 ms 02:32:32,600 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting for more work 02:32:32,600 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: persisted 1 entries in 7.306 ms 02:32:32,600 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting for more work 02:32:32,600 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: running 1 completion(s) 02:32:32,600 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower797: running 1 completion(s) 02:32:32,600 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower796: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:32,600 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower797: persist complete JournaledLogEntry{index=0, term=6, command=zero, pending=false} 02:32:32,600 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower796: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@49a9d910 02:32:32,600 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower797: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@1ab00d98 [WARN] [06/27/2025 14:32:32.601] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower796] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower796, initialSyncDone=true] [WARN] [06/27/2025 14:32:32.601] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower797] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower797, initialSyncDone=true] 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): Commit index set to 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower797 (Follower): Commit index set to 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower796 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower797 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower796 (Follower): Setting last applied to 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from -1 to 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorBehavior - follower797 (Follower): Setting last applied to 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractReplicatedLog - follower797: Moving last applied index from -1 to 0 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower796: Applying state for log index 0 data zero 02:32:32,787 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower796: applyState called: zero 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - follower797: Applying state for log index 0 data zero 02:32:32,787 PM [test-pekko.actor.default-dispatcher-12] [INFO] MockRaftActor - follower797: applyState called: zero 02:32:32,788 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 entries in 187.5 ms 02:32:32,788 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: persisted 1 entries in 319.3 μs 02:32:32,788 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting for more work 02:32:32,788 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 entries in 188.1 ms 02:32:32,789 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: persisted 1 entries in 81.85 μs 02:32:32,789 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: waiting for more work 02:32:32,907 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower797: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:32,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:33,27 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:33,146 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:33,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:33,352 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActor - leader795: Persist data index=1 term=6 command=one 02:32:33,352 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - leader795: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:33,352 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 entries in 752.1 ms 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower797: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): sendAppendEntries: 1 is present for follower follower796 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractLeader - leader795 (Leader): Sending AppendEntries to follower follower796: AppendEntries{term=6, leaderId=leader795, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): handleAppendEntries: AppendEntries{term=6, leaderId=leader795, prevLogIndex=0, prevLogTerm=6, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=6, command=one, pending=true}]} 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Number of entries to be appended = 1 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Append entry to log one 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower796: Append log entry and persist JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): Log size is now 2 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] Follower - follower796 (Follower): handleAppendEntries returning : AppendEntriesReply{term=6, success=true, followerId=follower796, logLastIndex=1, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 0 (of expected -1) 02:32:33,353 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 entries in 565.1 ms 02:32:33,353 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower796: awaiting more completions to resolve 1 deferred callback(s) 02:32:33,353 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower796 updated: matchIndex: 1, nextIndex: 2 02:32:33,354 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader795 (Leader): Checking sendAppendEntries for follower follower796: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:32:33,358 PM [leader795-writer-179] [DEBUG] EntryStoreCompleter - leader795: 1 completion(s) pending 02:32:33,358 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: persisted 1 entries in 5.841 ms 02:32:33,358 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting for more work 02:32:33,358 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - leader795: running 1 completion(s) 02:32:33,358 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader795: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=true} 02:32:33,358 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader795 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:32:33,359 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader795 (Leader): Setting last applied to 1 02:32:33,359 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader795: Moving last applied index from 0 to 1 02:32:33,359 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader795: Applying state for log index 1 data one 02:32:33,359 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader795: applyState called: one 02:32:33,359 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader795: fakeSnapshot purging log to 0 for term 6 02:32:33,359 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 entries in 832.5 μs 02:32:33,359 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: persisted 1 entries in 120.9 μs 02:32:33,359 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: waiting for more work 02:32:33,359 PM [follower796-writer-177] [DEBUG] EntryStoreCompleter - follower796: 1 completion(s) pending 02:32:33,359 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower796: running 1 completion(s) 02:32:33,359 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: persisted 1 entries in 5.994 ms 02:32:33,359 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting for more work 02:32:33,359 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] ReplicatedLogImpl - follower796: persist complete JournaledLogEntry{index=1, term=6, command=one, pending=false} 02:32:33,359 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] EntryStoreCompleter - follower796: completed deferred callback org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c087fa30@2448a306 02:32:33,506 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower796 (Follower): Commit index set to 1 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower796 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower796 (Follower): Setting last applied to 1 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower796: Moving last applied index from 0 to 1 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower796: Applying state for log index 1 data one 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower796: applyState called: one 02:32:33,507 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower796: fakeSnapshot purging log to 0 for term 6 02:32:33,507 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 entries in 147.6 ms 02:32:33,507 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: persisted 1 entries in 199.3 μs 02:32:33,507 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: waiting for more work 02:32:33,627 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:33,746 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:33,867 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:33,987 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,106 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,226 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,347 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,466 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,587 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,706 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,827 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:34,946 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,67 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,187 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,307 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,427 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,546 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,667 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,786 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:35,907 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,26 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,147 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,386 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,507 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,627 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,747 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,867 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:36,986 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,107 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,227 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,351 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,478 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,596 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,716 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:37,957 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:38,77 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:38,197 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:38,317 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower796: Lagging 0 entries behind leader leader795 and reached 1 (of expected -1) 02:32:38,403 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower796-collector798#102472433] 02:32:38,414 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower796] 02:32:38,414 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower796: terminating thread VirtualThread[#916,follower796-writer-177]/waiting 02:32:38,415 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: received 1 entries in 4.907 s 02:32:38,415 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: persisted 1 entries in 46.40 μs 02:32:38,415 PM [follower796-writer-177] [DEBUG] JournalWriteTask - follower796: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@7056f707 stopped 02:32:38,415 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower796: journal closed 02:32:38,415 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower796: journal closed 02:32:38,415 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: stopped executor with 0 remaining tasks 02:32:38,415 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower796: stopped executor with 0 remaining tasks 02:32:38,415 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower797-collector799#-1050222187] 02:32:38,416 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower797] 02:32:38,416 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - follower797: terminating thread VirtualThread[#917,follower797-writer-178]/waiting 02:32:38,416 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: received 1 entries in 5.627 s 02:32:38,416 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: persisted 1 entries in 14.72 μs 02:32:38,416 PM [follower797-writer-178] [DEBUG] JournalWriteTask - follower797: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@11278b04 stopped 02:32:38,416 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - follower797: journal closed 02:32:38,416 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - follower797: journal closed 02:32:38,416 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower797: stopped executor with 0 remaining tasks 02:32:38,416 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower797: stopped executor with 0 remaining tasks 02:32:38,416 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader795-collector800#-1364460384] 02:32:38,416 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader795] 02:32:38,417 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRaftStorage - leader795: terminating thread VirtualThread[#918,leader795-writer-179]/waiting 02:32:38,417 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: received 1 entries in 5.057 s 02:32:38,417 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: persisted 1 entries in 10.17 μs 02:32:38,417 PM [leader795-writer-179] [DEBUG] JournalWriteTask - leader795: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@573154e2 stopped 02:32:38,417 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader795: journal closed 02:32:38,417 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader795: journal closed 02:32:38,417 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader795: stopped executor with 0 remaining tasks 02:32:38,417 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader795: stopped executor with 0 remaining tasks [INFO] [06/27/2025 14:32:38.420] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 02:32:38,421 PM [test-pekko.actor.default-dispatcher-6] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [WARN] [06/27/2025 14:32:38.422] [test-pekko.actor.default-dispatcher-6] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.170.8:17355) [INFO] [06/27/2025 14:32:38.423] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Exiting completed 02:32:38,423 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Exiting completed [INFO] [06/27/2025 14:32:38.423] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Shutting down... 02:32:38,423 PM [test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Shutting down... [INFO] [06/27/2025 14:32:38.423] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Successfully shut down 02:32:38,424 PM [test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Successfully shut down [INFO] [06/27/2025 14:32:38.426] [test-pekko.remote.default-remote-dispatcher-9] [pekko://test@10.30.170.8:17355/system/remoting-terminator] Shutting down remote daemon. 02:32:38,426 PM [test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:32:38,426 PM [test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [INFO] [06/27/2025 14:32:38.426] [test-pekko.remote.default-remote-dispatcher-9] [pekko://test@10.30.170.8:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports. [INFO] [06/27/2025 14:32:38.428] [test-pekko.remote.default-remote-dispatcher-9] [pekko://test@10.30.170.8:17355/system/remoting-terminator] Remoting shut down. [DEBUG] [06/27/2025 14:32:38.428] [test-pekko.actor.internal-dispatcher-2] [EventStream] shutting down: StandardOutLogger 02:32:38,429 PM [test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down.