[DEBUG] [01/29/2025 22:39:54.500] [main] [EventStream(pekko://test)] logger log1-TestEventListener started 10:39:54,501 PM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started [DEBUG] [01/29/2025 22:39:54.501] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started [DEBUG] [01/29/2025 22:39:54.503] [main] [EventStream(pekko://test)] Default Loggers started [INFO] [01/29/2025 22:39:54.511] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.31:17355] with UID [-442883516880773322] 10:39:54,512 PM [test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.31:17355] with UID [-442883516880773322] [INFO] [01/29/2025 22:39:54.512] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - Starting up, Pekko version [1.0.3] ... 10:39:54,512 PM [test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - Starting up, Pekko version [1.0.3] ... [INFO] [01/29/2025 22:39:54.514] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 10:39:54,514 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 10:39:54,514 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - Started up successfully 10:39:54,515 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 10:39:54,515 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining [INFO] [01/29/2025 22:39:54.514] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - Started up successfully [INFO] [01/29/2025 22:39:54.514] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing [INFO] [01/29/2025 22:39:54.515] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 10:39:54,524 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferAborted starting 10:39:54,524 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/29/2025 22:39:54.526] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower551]: org.opendaylight.controller.cluster.common.actor.Monitor@12a1e347 10:39:54,526 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower551] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.526] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower551] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,528 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower551 with journal batch size 1000 [WARN] [01/29/2025 22:39:54.530] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower552]: org.opendaylight.controller.cluster.common.actor.Monitor@2379b716 [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.530] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower552] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,530 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower552] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,530 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower552 with journal batch size 1000 [WARN] [01/29/2025 22:39:54.531] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower553]: org.opendaylight.controller.cluster.common.actor.Monitor@50d3142b 10:39:54,532 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower553 with journal batch size 1000 [WARN] [01/29/2025 22:39:54.533] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader550]: org.opendaylight.controller.cluster.common.actor.Monitor@7d2a9ae2 10:39:54,533 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader550 with journal batch size 1000 10:39:54,534 PM [test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower553] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,534 PM [test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader550] to Actor[pekko://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.534] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower553] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.534] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader550] to Actor[pekko://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,535 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower553: SnapshotOffer called. 10:39:54,535 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: SnapshotOffer called. 10:39:54,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: SnapshotOffer called. 10:39:54,535 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Moving last applied index from -1 to -1 10:39:54,535 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower553: Moving last applied index from -1 to -1 10:39:54,535 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower552: Set currentTerm=1, votedFor=null 10:39:54,535 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PersistenceTermInfoStore - follower553: Set currentTerm=1, votedFor=null 10:39:54,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Moving last applied index from -1 to -1 10:39:54,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=1, votedFor=null 10:39:54,535 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower553: Updated server config: isVoting: false, peers: [PeerInfo [id=follower552, address=pekko://test/user/follower552, votingState=VOTING], PeerInfo [id=leader550, address=pekko://test/user/leader550, votingState=VOTING], PeerInfo [id=follower551, address=pekko://test/user/follower551, votingState=VOTING]] 10:39:54,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Updated server config: isVoting: true, peers: [PeerInfo [id=follower553, address=pekko://test/user/follower553, votingState=NON_VOTING], PeerInfo [id=follower552, address=pekko://test/user/follower552, votingState=VOTING], PeerInfo [id=leader550, address=pekko://test/user/leader550, votingState=VOTING]] 10:39:54,536 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Updated server config: isVoting: true, peers: [PeerInfo [id=follower553, address=pekko://test/user/follower553, votingState=NON_VOTING], PeerInfo [id=leader550, address=pekko://test/user/leader550, votingState=VOTING], PeerInfo [id=follower551, address=pekko://test/user/follower551, votingState=VOTING]] 10:39:54,535 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower551 in 70.82 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:54,535 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower553 in 60.08 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:54,536 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower552 in 192.4 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:54,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: SnapshotOffer called. 10:39:54,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: Moving last applied index from -1 to -1 10:39:54,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PersistenceTermInfoStore - leader550: Set currentTerm=1, votedFor=null 10:39:54,536 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: Updated server config: isVoting: true, peers: [PeerInfo [id=follower553, address=pekko://test/user/follower553, votingState=NON_VOTING], PeerInfo [id=follower552, address=pekko://test/user/follower552, votingState=VOTING], PeerInfo [id=follower551, address=pekko://test/user/follower551, votingState=VOTING]] 10:39:54,536 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader550 in 28.84 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:54,537 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Recovery completed in 1.119 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:54,537 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower553: Recovery completed in 1.138 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:54,537 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Recovery completed in 1.315 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:54,537 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: Recovery completed in 1.390 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.31:17355, Removed) 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Follower): Received ElectionTimeout - switching to Candidate 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate): Election: Candidate has following voting peers: [follower552, follower551] 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader550: Set currentTerm=2, votedFor=leader550 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate): Starting new election term 2 10:39:54,917 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate): Sending RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1} to peer follower552 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): Cluster state: [] 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower552: Set currentTerm=2, votedFor=null 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): In requestVote: RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower552: Set currentTerm=2, votedFor=leader550 10:39:54,918 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:54,918 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate): Sending RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1} to peer follower551 10:39:54,918 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:54,918 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Cluster state: [] 10:39:54,918 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:54,918 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:54,918 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=2, votedFor=null 10:39:54,919 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): In requestVote: RequestVote{term=2, candidateId=leader550, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:54,919 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=2, votedFor=leader550 10:39:54,919 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:54,919 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 10:39:54,920 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 10:39:54,920 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 10:39:54,920 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Election: Leader has following peers: [follower553, follower552, follower551] 10:39:54,920 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower553: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,920 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower552: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,921 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower551: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,921 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower553 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader550, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 10:39:54,921 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower553: Set currentTerm=2, votedFor=null 10:39:54,921 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 10:39:54,921 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower553: Last sync leader does not match current leader leader550, need to catch up to -1 10:39:54,921 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower551: Last sync leader does not match current leader leader550, need to catch up to -1 10:39:54,921 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower552: Last sync leader does not match current leader leader550, need to catch up to -1 [WARN] [01/29/2025 22:39:54.921] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower551] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@63211469 [WARN] [01/29/2025 22:39:54.921] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower552] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@45080697 [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.921] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower551] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.921] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower552] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,921 PM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,921 PM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [WARN] [01/29/2025 22:39:54.922] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower553] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@32983ca4 [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.922] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower553] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower553] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,922 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower553 updated: matchIndex: -1, nextIndex: 0 10:39:54,922 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower553: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,922 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower551 updated: matchIndex: -1, nextIndex: 0 10:39:54,922 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower551: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,922 PM [test-pekko.actor.default-dispatcher-5] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower553] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,922 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower552 updated: matchIndex: -1, nextIndex: 0 10:39:54,922 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower552: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,939 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:54,939 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader550 starting 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: onShutDown 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: Initiating leader transfer 10:39:54,942 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower553: onLeaderTransitioning: LeaderTransitioning [leaderId=leader550] 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - leader550: pauseLeader successfully completed - doing transfer 10:39:54,942 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: onLeaderTransitioning: LeaderTransitioning [leaderId=leader550] 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Attempting to transfer leadership 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower553: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,942 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: onLeaderTransitioning: LeaderTransitioning [leaderId=leader550] 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower552: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,942 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Checking sendAppendEntries for follower follower551: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:54,942 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower553: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:54,943 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:54.943] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower553] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@58a3efe2 [WARN] [01/29/2025 22:39:54.943] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower551] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@74ac2efe [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.943] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower553] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower553] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.943] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower551] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,943 PM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower553] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,943 PM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,943 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:54.943] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower552] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@d57dcc0 [INFO] [pekkoDeadLetter][01/29/2025 22:39:54.943] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower552] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:54,943 PM [test-pekko.actor.default-dispatcher-18] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 10:39:55,38 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,38 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,38 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower553: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,158 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,158 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,158 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower553: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - leader550: leader transfer aborted 10:39:55,277 PM [test-pekko.actor.default-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - leader550: Failed to transfer leadership in 335.3 ms 10:39:55,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550: leader transfer failed - sending PoisonPill 10:39:55,277 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader550 (Leader): Leadership transfer expired 10:39:55,278 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,278 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:55.278] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader550] received dead letter from TestActor[pekko://test/user/follower552]: AppendEntriesReply{term=2, success=true, followerId=follower552, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} [WARN] [01/29/2025 22:39:55.278] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader550] received dead letter from TestActor[pekko://test/user/follower551]: AppendEntriesReply{term=2, success=true, followerId=follower551, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} [WARN] [01/29/2025 22:39:55.278] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader550] received dead letter from TestActor[pekko://test/user/follower553]: AppendEntriesReply{term=2, success=true, followerId=follower553, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 10:39:55,278 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower553: Lagging 0 entries behind leader leader550 and reached -1 (of expected -1) 10:39:55,279 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader550 ending 10:39:55,281 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferOnShutDown ending 10:39:55,281 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower551-notifier554#-609428738] 10:39:55,292 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower551-collector555#1081149914] 10:39:55,292 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower551] 10:39:55,293 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower552-notifier556#526859979] 10:39:55,293 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower552-collector557#-1507730915] 10:39:55,293 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower552] 10:39:55,293 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower553-notifier558#-384994814] 10:39:55,294 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower553-collector559#597525098] 10:39:55,294 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower553] 10:39:55,294 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader550-notifier560#1188693687] 10:39:55,294 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader550-collector561#-378521497] 10:39:55,294 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader550] 10:39:55,295 PM [main] [INFO] LeadershipTransferIntegrationTest - testSuccessfulRequestLeadershipTransferToFollower2 starting 10:39:55,295 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/29/2025 22:39:55.298] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower563]: org.opendaylight.controller.cluster.common.actor.Monitor@71cbbb65 10:39:55,299 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower563 with journal batch size 1000 10:39:55,300 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: SnapshotOffer called. 10:39:55,300 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Moving last applied index from -1 to -1 10:39:55,300 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=1, votedFor=null 10:39:55,300 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Updated server config: isVoting: true, peers: [PeerInfo [id=leader562, address=pekko://test/user/leader562, votingState=VOTING], PeerInfo [id=follower564, address=pekko://test/user/follower564, votingState=VOTING], PeerInfo [id=follower565, address=pekko://test/user/follower565, votingState=NON_VOTING]] 10:39:55,300 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower563 in 130.4 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,301 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Recovery completed in 1.001 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:55.302] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower564]: org.opendaylight.controller.cluster.common.actor.Monitor@41186615 10:39:55,302 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower564 with journal batch size 1000 10:39:55,303 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: SnapshotOffer called. 10:39:55,303 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Moving last applied index from -1 to -1 10:39:55,303 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=1, votedFor=null 10:39:55,303 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Updated server config: isVoting: true, peers: [PeerInfo [id=leader562, address=pekko://test/user/leader562, votingState=VOTING], PeerInfo [id=follower563, address=pekko://test/user/follower563, votingState=VOTING], PeerInfo [id=follower565, address=pekko://test/user/follower565, votingState=NON_VOTING]] 10:39:55,303 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower564 in 47.61 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,303 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Recovery completed in 599.9 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:55.304] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower565]: org.opendaylight.controller.cluster.common.actor.Monitor@38e37ea0 10:39:55,304 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower565 with journal batch size 1000 10:39:55,305 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower565: SnapshotOffer called. 10:39:55,305 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower565: Moving last applied index from -1 to -1 10:39:55,305 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower565: Set currentTerm=1, votedFor=null 10:39:55,305 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower565: Updated server config: isVoting: false, peers: [PeerInfo [id=leader562, address=pekko://test/user/leader562, votingState=VOTING], PeerInfo [id=follower564, address=pekko://test/user/follower564, votingState=VOTING], PeerInfo [id=follower563, address=pekko://test/user/follower563, votingState=VOTING]] [WARN] [01/29/2025 22:39:55.305] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader562]: org.opendaylight.controller.cluster.common.actor.Monitor@5ee99d88 10:39:55,306 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader562 with journal batch size 1000 10:39:55,305 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower565 in 227.4 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,306 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: SnapshotOffer called. 10:39:55,306 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: Moving last applied index from -1 to -1 10:39:55,306 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - leader562: Set currentTerm=1, votedFor=null 10:39:55,306 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: Updated server config: isVoting: true, peers: [PeerInfo [id=follower564, address=pekko://test/user/follower564, votingState=VOTING], PeerInfo [id=follower563, address=pekko://test/user/follower563, votingState=VOTING], PeerInfo [id=follower565, address=pekko://test/user/follower565, votingState=NON_VOTING]] 10:39:55,306 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader562 in 42.13 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,307 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: Recovery completed in 700.4 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:55,307 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower565: Recovery completed in 2.109 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:55,657 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:55,657 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.31:17355, Removed) 10:39:55,657 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Follower): Received ElectionTimeout - switching to Candidate 10:39:55,657 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate): Election: Candidate has following voting peers: [follower564, follower563] 10:39:55,657 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - leader562: Set currentTerm=2, votedFor=leader562 10:39:55,658 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate): Starting new election term 2 10:39:55,658 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate): Sending RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1} to peer follower564 10:39:55,658 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate): Sending RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1} to peer follower563 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Cluster state: [] 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Cluster state: [] 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=2, votedFor=null 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=2, votedFor=null 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): In requestVote: RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=2, votedFor=leader562 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): In requestVote: RequestVote{term=2, candidateId=leader562, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=2, votedFor=leader562 10:39:55,658 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:55,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:55,658 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Election: Leader has following peers: [follower564, follower563, follower565] 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower564: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower563: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower565: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,660 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 10:39:55,660 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower563: Last sync leader does not match current leader leader562, need to catch up to -1 10:39:55,660 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower565 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader562, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 10:39:55,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower565: Set currentTerm=2, votedFor=null 10:39:55,660 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower565: Last sync leader does not match current leader leader562, need to catch up to -1 10:39:55,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower563 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/29/2025 22:39:55.661] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower563] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@10cf00dc 10:39:55,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower565 updated: matchIndex: -1, nextIndex: 0 10:39:55,661 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower564: Last sync leader does not match current leader leader562, need to catch up to -1 10:39:55,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower565: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower564 updated: matchIndex: -1, nextIndex: 0 10:39:55,661 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:55.661] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower564] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@63b8a913 10:39:55,661 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:55,661 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting [WARN] [01/29/2025 22:39:55.661] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower565] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@5050daca 10:39:55,662 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 10:39:55,662 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 10:39:55,662 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 10:39:55,662 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: onRequestLeadership RequestLeadership [requestedFollowerId=follower564, replyTo=Actor[pekko://test/user/$a#-313734270]] 10:39:55,664 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: Initiating leader transfer 10:39:55,664 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorLeadershipTransferCohort - leader562: pauseLeader successfully completed - doing transfer 10:39:55,664 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Attempting to transfer leadership 10:39:55,664 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: onLeaderTransitioning: LeaderTransitioning [leaderId=leader562] 10:39:55,664 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: onLeaderTransitioning: LeaderTransitioning [leaderId=leader562] 10:39:55,664 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,664 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower565: onLeaderTransitioning: LeaderTransitioning [leaderId=leader562] 10:39:55,664 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower564: Lagging 0 entries behind leader leader562 and reached -1 (of expected -1) 10:39:55,665 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:55.665] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower564] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@770be256 10:39:55,665 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower563: Lagging 0 entries behind leader leader562 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:55.665] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower563] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@76c0023f 10:39:55,665 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower565: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,665 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower565: Lagging 0 entries behind leader leader562 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:55.665] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower565] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@216246ea 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): tryToCompleteLeadershipTransfer: followerId: follower564, matchIndex: -1, lastIndex: -1, isVoting: true 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Follower's log matches - sending ElectionTimeout 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,666 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower564: Lagging 0 entries behind leader leader562 and reached -1 (of expected -1) 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Checking sendAppendEntries for follower follower565: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,666 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower563: Lagging 0 entries behind leader leader562 and reached -1 (of expected -1) 10:39:55,666 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower565: Lagging 0 entries behind leader leader562 and reached -1 (of expected -1) 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Leader transfer complete 10:39:55,666 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorLeadershipTransferCohort - leader562: leader transfer complete - waiting for new leader 10:39:55,666 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:55,666 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Received TimeoutNow - switching to Candidate 10:39:55,667 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate): Election: Candidate has following voting peers: [leader562, follower563] 10:39:55,667 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=3, votedFor=follower564 10:39:55,667 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate): Starting new election term 3 10:39:55,667 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate): Sending RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1} to peer leader562 10:39:55,667 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate): Sending RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1} to peer follower563 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:55,667 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower) :- Switching from behavior Follower to Candidate, election term: 3 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Cluster state: [] 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Term 3 in "RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 2 - updating term 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=3, votedFor=null 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): In requestVote: RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=3, votedFor=follower564 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Cluster state: [] 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Term 3 in "RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1}" message is greater than leader's term 2 - switching to Follower 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader562: Set currentTerm=3, votedFor=null 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): Leadership transfer in progress - processing RequestVote 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): In requestVote: RequestVote{term=3, candidateId=follower564, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader562: Set currentTerm=3, votedFor=follower564 10:39:55,667 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 10:39:55,668 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader562 (Leader) :- Switching from behavior Leader to Follower, election term: 3 10:39:55,668 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 10:39:55,668 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Election: Leader has following peers: [leader562, follower563, follower565] 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Checking sendAppendEntries for follower leader562: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Checking sendAppendEntries for follower follower563: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Checking sendAppendEntries for follower follower565: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 10:39:55,669 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower563: Last sync leader does not match current leader follower564, need to catch up to -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower563 updated: matchIndex: -1, nextIndex: 0 10:39:55,669 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:55.669] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower563] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3d6b73a7 10:39:55,669 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower565 (Follower): Term 3 in "AppendEntries{term=3, leaderId=follower564, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 10:39:55,669 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower565: Set currentTerm=3, votedFor=null 10:39:55,669 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - leader562: Last sync leader does not match current leader follower564, need to catch up to -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - leader562: leader changed to follower564 10:39:55,669 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): handleAppendEntriesReply - FollowerLogInformation for leader562 updated: matchIndex: -1, nextIndex: 0 10:39:55,669 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Checking sendAppendEntries for follower leader562: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorLeadershipTransferCohort - leader562: Successfully transferred leadership to follower564 in 5.043 ms 10:39:55,669 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader562: Leadership transferred successfully to follower564 10:39:55,669 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower565: Last sync leader does not match current leader follower564, need to catch up to -1 10:39:55,669 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower565 updated: matchIndex: -1, nextIndex: 0 10:39:55,669 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Leader): Checking sendAppendEntries for follower follower565: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:55.669] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader562] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@127c6ac0 [WARN] [01/29/2025 22:39:55.670] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower565] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@7f205ece 10:39:55,714 PM [main] [INFO] LeadershipTransferIntegrationTest - testSuccessfulRequestLeadershipTransferToFollower2 ending 10:39:55,714 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower563-notifier566#1574980835] 10:39:55,725 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower563-collector567#156338864] 10:39:55,725 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower563] 10:39:55,725 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower564-notifier568#-1137717005] 10:39:55,725 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower564-collector569#1131159904] 10:39:55,725 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower564] 10:39:55,726 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower565-notifier570#1262513606] 10:39:55,726 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower565-collector571#-326189974] 10:39:55,726 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower565] 10:39:55,726 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader562-notifier572#-1655525006] 10:39:55,726 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader562-collector573#-1312915979] 10:39:55,726 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader562] 10:39:55,727 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$a#-313734270] 10:39:55,728 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Shutdown starting 10:39:55,728 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:55,730 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower575 with journal batch size 1000 [WARN] [01/29/2025 22:39:55.730] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower575]: org.opendaylight.controller.cluster.common.actor.Monitor@1ca946b2 10:39:55,731 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: SnapshotOffer called. 10:39:55,731 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Moving last applied index from -1 to -1 10:39:55,731 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=1, votedFor=null 10:39:55,731 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Updated server config: isVoting: true, peers: [PeerInfo [id=leader574, address=pekko://test/user/leader574, votingState=VOTING], PeerInfo [id=follower577, address=pekko://test/user/follower577, votingState=NON_VOTING], PeerInfo [id=follower576, address=pekko://test/user/follower576, votingState=VOTING]] 10:39:55,731 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower575 in 138.0 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,733 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Recovery completed in 1.201 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:55.733] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower576]: org.opendaylight.controller.cluster.common.actor.Monitor@1d542696 10:39:55,733 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower576 with journal batch size 1000 10:39:55,734 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: SnapshotOffer called. 10:39:55,734 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Moving last applied index from -1 to -1 10:39:55,734 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower576: Set currentTerm=1, votedFor=null 10:39:55,734 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Updated server config: isVoting: true, peers: [PeerInfo [id=leader574, address=pekko://test/user/leader574, votingState=VOTING], PeerInfo [id=follower575, address=pekko://test/user/follower575, votingState=VOTING], PeerInfo [id=follower577, address=pekko://test/user/follower577, votingState=NON_VOTING]] 10:39:55,734 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower576 in 65.48 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,735 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Recovery completed in 614.7 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:55.735] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower577]: org.opendaylight.controller.cluster.common.actor.Monitor@2531afcf 10:39:55,736 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower577 with journal batch size 1000 10:39:55,736 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower577: SnapshotOffer called. 10:39:55,736 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower577: Moving last applied index from -1 to -1 10:39:55,736 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower577: Set currentTerm=1, votedFor=null 10:39:55,736 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower577: Updated server config: isVoting: false, peers: [PeerInfo [id=leader574, address=pekko://test/user/leader574, votingState=VOTING], PeerInfo [id=follower575, address=pekko://test/user/follower575, votingState=VOTING], PeerInfo [id=follower576, address=pekko://test/user/follower576, votingState=VOTING]] 10:39:55,736 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower577 in 40.30 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,737 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower577: Recovery completed in 530.5 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:55.738] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader574]: org.opendaylight.controller.cluster.common.actor.Monitor@4d722a3a 10:39:55,738 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader574 with journal batch size 1000 10:39:55,738 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: SnapshotOffer called. 10:39:55,738 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: Moving last applied index from -1 to -1 10:39:55,738 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - leader574: Set currentTerm=1, votedFor=null 10:39:55,739 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: Updated server config: isVoting: true, peers: [PeerInfo [id=follower575, address=pekko://test/user/follower575, votingState=VOTING], PeerInfo [id=follower577, address=pekko://test/user/follower577, votingState=NON_VOTING], PeerInfo [id=follower576, address=pekko://test/user/follower576, votingState=VOTING]] 10:39:55,739 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader574 in 62.02 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:55,739 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: Recovery completed in 467.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.31:17355, Removed) 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Follower): Received ElectionTimeout - switching to Candidate 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate): Election: Candidate has following voting peers: [follower575, follower576] 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader574: Set currentTerm=2, votedFor=leader574 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate): Starting new election term 2 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate): Sending RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1} to peer follower575 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate): Sending RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1} to peer follower576 10:39:56,127 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 10:39:56,127 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:56,127 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Cluster state: [] 10:39:56,127 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:56,127 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:56,127 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:56,127 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=2, votedFor=null 10:39:56,127 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): Cluster state: [] 10:39:56,127 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:56,127 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:56,127 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower576: Set currentTerm=2, votedFor=null 10:39:56,127 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): In requestVote: RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:56,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=2, votedFor=leader574 10:39:56,128 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): In requestVote: RequestVote{term=2, candidateId=leader574, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:56,128 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower576: Set currentTerm=2, votedFor=leader574 10:39:56,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:56,128 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:56,128 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 10:39:56,128 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Election: Leader has following peers: [follower575, follower577, follower576] 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower575: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower577: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower576: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 10:39:56,129 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower575: Last sync leader does not match current leader leader574, need to catch up to -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower577 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader574, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 10:39:56,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower577: Set currentTerm=2, votedFor=null 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower575 updated: matchIndex: -1, nextIndex: 0 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower575: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower577: Last sync leader does not match current leader leader574, need to catch up to -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower577 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/29/2025 22:39:56.129] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower575] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6c6ed02b 10:39:56,129 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower577: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower576: Last sync leader does not match current leader leader574, need to catch up to -1 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower576 updated: matchIndex: -1, nextIndex: 0 10:39:56,129 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower576: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:56.129] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower576] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6d1ddbfe [WARN] [01/29/2025 22:39:56.130] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower577] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@4a27b9e0 10:39:56,142 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:56,142 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 10:39:56,142 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 10:39:56,142 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower576 starting 10:39:56,142 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: onShutDown 10:39:56,143 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower576 ending 10:39:56,143 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 10:39:56,143 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 10:39:56,143 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: onRequestLeadership RequestLeadership [requestedFollowerId=follower576, replyTo=Actor[pekko://test/user/$b#2063218512]] 10:39:56,143 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: Initiating leader transfer 10:39:56,143 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: onLeaderTransitioning: LeaderTransitioning [leaderId=leader574] 10:39:56,143 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower577: onLeaderTransitioning: LeaderTransitioning [leaderId=leader574] 10:39:56,143 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - leader574: pauseLeader successfully completed - doing transfer 10:39:56,143 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Attempting to transfer leadership [WARN] [01/29/2025 22:39:56.143] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower576] received dead letter from TestActor[pekko://test/user/leader574]: LeaderTransitioning [leaderId=leader574] 10:39:56,143 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower575: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,143 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower577: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,143 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower575: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) 10:39:56,144 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Checking sendAppendEntries for follower follower576: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:56.144] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower575] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3aefd033 10:39:56,144 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower577: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:56.144] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower576] received dead letter from TestActor[pekko://test/user/leader574]: AppendEntries{term=2, leaderId=leader574, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [01/29/2025 22:39:56.144] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower577] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@39ae3ebb 10:39:56,247 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower577: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:56.247] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower576] received dead letter from TestActor[pekko://test/user/leader574]: AppendEntries{term=2, leaderId=leader574, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:56,247 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower575: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) 10:39:56,367 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower575: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) 10:39:56,368 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower577: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:56.368] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower576] received dead letter from TestActor[pekko://test/user/leader574]: AppendEntries{term=2, leaderId=leader574, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:56,488 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorLeadershipTransferCohort - leader574: leader transfer aborted 10:39:56,488 PM [test-pekko.actor.default-dispatcher-6] [WARN] RaftActorLeadershipTransferCohort - leader574: Failed to transfer leadership in 344.7 ms 10:39:56,488 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574: LeadershipTransfer request from follower576 failed 10:39:56,489 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader574 (Leader): Leadership transfer expired 10:39:56,489 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower575: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) 10:39:56,489 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower577: Lagging 0 entries behind leader leader574 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:56.489] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower576] received dead letter from TestActor[pekko://test/user/leader574]: AppendEntries{term=2, leaderId=leader574, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:56,496 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Shutdown ending 10:39:56,496 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower575-notifier578#1724457850] 10:39:56,497 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower575-collector579#-2029072618] 10:39:56,498 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower575] 10:39:56,498 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower576-notifier580#975468744] 10:39:56,498 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower576-collector581#912267031] 10:39:56,498 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower576] 10:39:56,499 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower577-notifier582#1010000238] 10:39:56,499 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower577-collector583#-1257776567] 10:39:56,499 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower577] 10:39:56,499 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader574-notifier584#604056234] 10:39:56,499 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader574-collector585#-1521457227] 10:39:56,500 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader574] 10:39:56,500 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$b#2063218512] 10:39:56,501 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferOnShutDown starting 10:39:56,501 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/29/2025 22:39:56.502] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower587]: org.opendaylight.controller.cluster.common.actor.Monitor@2f6ee954 10:39:56,503 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower587 with journal batch size 1000 10:39:56,503 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: SnapshotOffer called. 10:39:56,503 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Moving last applied index from -1 to -1 10:39:56,503 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=1, votedFor=null 10:39:56,503 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Updated server config: isVoting: true, peers: [PeerInfo [id=leader586, address=pekko://test/user/leader586, votingState=VOTING], PeerInfo [id=follower589, address=pekko://test/user/follower589, votingState=NON_VOTING], PeerInfo [id=follower588, address=pekko://test/user/follower588, votingState=VOTING]] 10:39:56,503 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower587 in 90.96 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/29/2025 22:39:56.504] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower588]: org.opendaylight.controller.cluster.common.actor.Monitor@2c0ba7b 10:39:56,504 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower588 with journal batch size 1000 10:39:56,504 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Recovery completed in 1.073 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:56,505 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: SnapshotOffer called. 10:39:56,505 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Moving last applied index from -1 to -1 10:39:56,505 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=1, votedFor=null 10:39:56,505 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Updated server config: isVoting: true, peers: [PeerInfo [id=leader586, address=pekko://test/user/leader586, votingState=VOTING], PeerInfo [id=follower589, address=pekko://test/user/follower589, votingState=NON_VOTING], PeerInfo [id=follower587, address=pekko://test/user/follower587, votingState=VOTING]] 10:39:56,505 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower588 in 118.2 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:56,506 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Recovery completed in 901.2 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:56.506] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower589]: org.opendaylight.controller.cluster.common.actor.Monitor@24908208 10:39:56,507 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower589 with journal batch size 1000 10:39:56,507 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: SnapshotOffer called. 10:39:56,508 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Moving last applied index from -1 to -1 10:39:56,508 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower589: Set currentTerm=1, votedFor=null 10:39:56,508 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Updated server config: isVoting: false, peers: [PeerInfo [id=leader586, address=pekko://test/user/leader586, votingState=VOTING], PeerInfo [id=follower588, address=pekko://test/user/follower588, votingState=VOTING], PeerInfo [id=follower587, address=pekko://test/user/follower587, votingState=VOTING]] 10:39:56,508 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower589 in 66.07 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/29/2025 22:39:56.508] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader586]: org.opendaylight.controller.cluster.common.actor.Monitor@3c41dfed 10:39:56,509 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Recovery completed in 863.2 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:56,509 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader586 with journal batch size 1000 10:39:56,509 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: SnapshotOffer called. 10:39:56,509 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Moving last applied index from -1 to -1 10:39:56,509 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - leader586: Set currentTerm=1, votedFor=null 10:39:56,509 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Updated server config: isVoting: true, peers: [PeerInfo [id=follower589, address=pekko://test/user/follower589, votingState=NON_VOTING], PeerInfo [id=follower588, address=pekko://test/user/follower588, votingState=VOTING], PeerInfo [id=follower587, address=pekko://test/user/follower587, votingState=VOTING]] 10:39:56,509 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader586 in 61.09 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:56,510 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Recovery completed in 583.2 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:56,887 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:56,887 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.31:17355, Removed) 10:39:56,887 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Follower): Received ElectionTimeout - switching to Candidate 10:39:56,887 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate): Election: Candidate has following voting peers: [follower588, follower587] 10:39:56,888 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - leader586: Set currentTerm=2, votedFor=leader586 10:39:56,888 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate): Starting new election term 2 10:39:56,888 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate): Sending RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1} to peer follower588 10:39:56,888 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate): Sending RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1} to peer follower587 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Cluster state: [] 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Cluster state: [] 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=2, votedFor=null 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=2, votedFor=null 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): In requestVote: RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): In requestVote: RequestVote{term=2, candidateId=leader586, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=2, votedFor=leader586 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=2, votedFor=leader586 10:39:56,888 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:56,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:56,888 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Election: Leader has following peers: [follower589, follower588, follower587] 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower589: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower588: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower587: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 10:39:56,890 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 10:39:56,890 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower589: Set currentTerm=2, votedFor=null 10:39:56,890 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower587: Last sync leader does not match current leader leader586, need to catch up to -1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower589: Last sync leader does not match current leader leader586, need to catch up to -1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower587 updated: matchIndex: -1, nextIndex: 0 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,890 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower589 updated: matchIndex: -1, nextIndex: 0 10:39:56,891 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower589: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:56,891 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower588: Last sync leader does not match current leader leader586, need to catch up to -1 [WARN] [01/29/2025 22:39:56.891] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower587] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@4a44dcab 10:39:56,891 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower588 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/29/2025 22:39:56.891] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@7d36ca1c 10:39:56,891 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:56.891] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower589] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@5cf7a6c3 10:39:56,913 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:56,913 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging starting 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Persist data SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower589: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower589 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Sending AppendEntries to follower follower589: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower588 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Sending AppendEntries to follower follower588: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:56,913 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower587 10:39:56,913 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Number of entries to be appended = 1 10:39:56,913 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 10:39:56,913 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Sending AppendEntries to follower follower587: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:56,913 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Append entry to log zero 10:39:56,913 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:56,914 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Log size is now 1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower589, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:56,914 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower589: Lagging 0 entries behind leader leader586 and reached -1 (of expected -1) 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Number of entries to be appended = 1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Append entry to log zero 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Log size is now 1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower587, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower587: Lagging 0 entries behind leader leader586 and reached -1 (of expected -1) 10:39:56,914 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower589 updated: matchIndex: 0, nextIndex: 1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower589: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower587 updated: matchIndex: 0, nextIndex: 1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:56,914 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] [WARN] [01/29/2025 22:39:56.915] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower589] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@29dfdd8c 10:39:56,915 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Setting last applied to 0 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Moving last applied index from -1 to 0 [WARN] [01/29/2025 22:39:56.915] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower587] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@49b9f594 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Applying state for log index 0 data zero 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: applyState called: zero 10:39:56,915 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Persisting ApplyJournalEntries with index=0 10:39:57,7 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower588 10:39:57,8 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Sending AppendEntries to follower follower588: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:57,127 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower588 10:39:57,127 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Sending AppendEntries to follower follower588: AppendEntries{term=2, leaderId=leader586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:57,127 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Commit index set to 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower589: Lagging 0 entries behind leader leader586 and reached 0 (of expected -1) 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Commit index set to 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower587: Lagging 0 entries behind leader leader586 and reached 0 (of expected -1) 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Setting last applied to 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Moving last applied index from -1 to 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Setting last applied to 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Moving last applied index from -1 to 0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Applying state for log index 0 data zero 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Applying state for log index 0 data zero 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: applyState called: zero 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: applyState called: zero 10:39:57,128 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Persisting ApplyJournalEntries with index=0 10:39:57,128 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: Persisting ApplyJournalEntries with index=0 10:39:57,164 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging ending 10:39:57,164 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDownToLeaderAndVerifyLeadershipTransferToFollower1 starting 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: onShutDown 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: Initiating leader transfer 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - leader586: pauseLeader successfully completed - doing transfer 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Attempting to transfer leadership 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower589: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower588 10:39:57,165 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,165 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower589: onLeaderTransitioning: LeaderTransitioning [leaderId=leader586] 10:39:57,165 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: onLeaderTransitioning: LeaderTransitioning [leaderId=leader586] 10:39:57,165 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower587: Lagging 0 entries behind leader leader586 and reached 0 (of expected -1) 10:39:57,165 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower589: Lagging 0 entries behind leader leader586 and reached 0 (of expected -1) 10:39:57,165 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): tryToCompleteLeadershipTransfer: followerId: follower589, matchIndex: 0, lastIndex: 0, isVoting: false 10:39:57,165 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): tryToCompleteLeadershipTransfer: followerId: follower587, matchIndex: 0, lastIndex: 0, isVoting: true 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Follower's log matches - sending ElectionTimeout 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower589: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): sendAppendEntries: 0 is present for follower follower588 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower589: Lagging 0 entries behind leader leader586 and reached 0 (of expected -1) 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Leader transfer complete 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorLeadershipTransferCohort - leader586: leader transfer complete - waiting for new leader 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower587: Lagging 0 entries behind leader leader586 and reached 0 (of expected -1) 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Received TimeoutNow - switching to Candidate 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate): Election: Candidate has following voting peers: [leader586, follower588] 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=3, votedFor=follower587 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate): Starting new election term 3 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate): Sending RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2} to peer leader586 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate): Sending RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2} to peer follower588 10:39:57,166 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower) :- Switching from behavior Follower to Candidate, election term: 3 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Cluster state: [] 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2} with higher term appears reachable, updating term. 10:39:57,166 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Term 3 in "RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2}" message is greater than leader's term 2 - switching to Follower 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader586: Set currentTerm=3, votedFor=null 10:39:57,166 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Cluster state: [] 10:39:57,166 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2} with higher term appears reachable, updating term. 10:39:57,166 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Term 3 in "RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2}" message is greater than follower's term 2 - updating term 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): Leadership transfer in progress - processing RequestVote 10:39:57,166 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=3, votedFor=null 10:39:57,166 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): In requestVote: RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2} - currentTerm: 3, votedFor: null, lastIndex: 0, lastTerm: 2 10:39:57,167 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader586: Set currentTerm=3, votedFor=follower587 10:39:57,167 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): In requestVote: RequestVote{term=3, candidateId=follower587, lastLogIndex=0, lastLogTerm=2} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:57,167 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 10:39:57,167 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=3, votedFor=follower587 10:39:57,167 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 10:39:57,167 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader586 (Leader) :- Switching from behavior Leader to Follower, election term: 3 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Election: Leader has following peers: [leader586, follower589, follower588] 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Checking sendAppendEntries for follower leader586: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Checking sendAppendEntries for follower follower589: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Checking sendAppendEntries for follower follower588: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,168 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower589 (Follower): Term 3 in "AppendEntries{term=3, leaderId=follower587, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 10:39:57,168 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower589: Set currentTerm=3, votedFor=null 10:39:57,168 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower589: Last sync leader does not match current leader follower587, need to catch up to -1 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower589 updated: matchIndex: 0, nextIndex: 1 10:39:57,168 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Checking sendAppendEntries for follower follower589: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,169 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): handleAppendEntriesReply - FollowerLogInformation for leader586 updated: matchIndex: 0, nextIndex: 1 10:39:57,169 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Checking sendAppendEntries for follower leader586: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,169 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower589: Lagging 0 entries behind leader follower587 and reached 0 (of expected -1) [WARN] [01/29/2025 22:39:57.169] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower589] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@5f4a8a0b 10:39:57,169 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - leader586: Last sync leader does not match current leader follower587, need to catch up to -1 [WARN] [01/29/2025 22:39:57.169] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower589] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@725a2f69 10:39:57,169 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorLeadershipTransferCohort - leader586: leader changed to follower587 10:39:57,170 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorLeadershipTransferCohort - leader586: Successfully transferred leadership to follower587 in 4.780 ms 10:39:57,170 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader586: leader transfer succeeded - sending PoisonPill 10:39:57,170 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - leader586: Lagging 0 entries behind leader follower587 and reached 0 (of expected -1) [WARN] [01/29/2025 22:39:57.170] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader586] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@202653f5 [WARN] [01/29/2025 22:39:57.170] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader586] received dead letter: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3fb20500 [WARN] [01/29/2025 22:39:57.688] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/leader586] received dead letter from TestActor[pekko://test/user/follower587]: AppendEntries{term=3, leaderId=follower587, prevLogIndex=0, prevLogTerm=2, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:57,688 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower589: Lagging 0 entries behind leader follower587 and reached 0 (of expected -1) 10:39:57,688 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower588: Last sync leader does not match current leader follower587, need to catch up to -1 10:39:57,688 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): sendAppendEntries: 0 is present for follower follower588 10:39:57,688 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Sending AppendEntries to follower follower588: AppendEntries{term=3, leaderId=follower587, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} [WARN] [01/29/2025 22:39:57.689] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@4e63bd3e 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=follower587, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Number of entries to be appended = 1 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Append entry to log zero 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Log size is now 1 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Commit index set to 0 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower588, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader follower587 and reached 0 (of expected -1) 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Setting last applied to 0 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Moving last applied index from -1 to 0 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Applying state for log index 0 data zero 10:39:57,689 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower588 updated: matchIndex: 0, nextIndex: 1 10:39:57,689 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:57,689 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: applyState called: zero 10:39:57,690 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:57,690 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Persisting ApplyJournalEntries with index=0 [WARN] [01/29/2025 22:39:57.690] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3e6f1905 10:39:57,720 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: onLeaderTransitioning: LeaderTransitioning [leaderId=leader586] 10:39:57,720 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDownToLeaderAndVerifyLeadershipTransferToFollower1 ending 10:39:57,720 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower588 starting 10:39:57,720 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: onShutDown 10:39:57,721 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower588 ending 10:39:57,721 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferOnShutDown ending 10:39:57,721 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower587-notifier590#453975814] 10:39:57,722 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower587-collector591#1492488581] 10:39:57,722 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower587] 10:39:57,722 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower588-notifier592#579957023] 10:39:57,723 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower588-collector593#1219007149] 10:39:57,723 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower588] 10:39:57,723 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower589-notifier594#-228796214] 10:39:57,723 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower589-collector595#1563913238] 10:39:57,723 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower589] 10:39:57,724 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader586-notifier596#-168082056] 10:39:57,724 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader586-collector597#304175295] 10:39:57,724 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader586] 10:39:57,725 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferSkippedOnShutdownWithNoFollowers starting [WARN] [01/29/2025 22:39:57.726] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader598]: org.opendaylight.controller.cluster.common.actor.Monitor@12a11981 10:39:57,727 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader598 with journal batch size 1000 10:39:57,728 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader598: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:57,728 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:57,728 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Follower): Received TimeoutNow - switching to Candidate 10:39:57,728 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Candidate): Election: Candidate has following voting peers: [] 10:39:57,728 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader598: Set currentTerm=1, votedFor=leader598 10:39:57,729 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Candidate): Starting new election term 1 10:39:57,729 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 10:39:57,729 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Candidate): Received ElectionTimeout 10:39:57,729 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Leader): Election: Leader has following peers: [] 10:39:57,729 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader598 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 10:39:57,729 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader598 starting 10:39:57,729 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader598: onShutDown 10:39:57,730 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader598 ending 10:39:57,730 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferSkippedOnShutdownWithNoFollowers ending 10:39:57,730 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader598-collector602#-2010371314] 10:39:57,730 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader598] 10:39:57,731 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Lagging starting 10:39:57,731 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/29/2025 22:39:57.732] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower604]: org.opendaylight.controller.cluster.common.actor.Monitor@7d2dd2e0 10:39:57,733 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower604 with journal batch size 1000 10:39:57,733 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: SnapshotOffer called. 10:39:57,733 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Moving last applied index from -1 to -1 10:39:57,733 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=1, votedFor=null 10:39:57,733 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Updated server config: isVoting: true, peers: [PeerInfo [id=follower605, address=pekko://test/user/follower605, votingState=VOTING], PeerInfo [id=follower606, address=pekko://test/user/follower606, votingState=NON_VOTING], PeerInfo [id=leader603, address=pekko://test/user/leader603, votingState=VOTING]] 10:39:57,733 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower604 in 74.51 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/29/2025 22:39:57.734] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower605]: org.opendaylight.controller.cluster.common.actor.Monitor@7becb2f5 10:39:57,734 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Recovery completed in 687.3 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:57,734 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower605 with journal batch size 1000 10:39:57,735 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: SnapshotOffer called. 10:39:57,735 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Moving last applied index from -1 to -1 10:39:57,735 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower605: Set currentTerm=1, votedFor=null 10:39:57,735 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Updated server config: isVoting: true, peers: [PeerInfo [id=follower604, address=pekko://test/user/follower604, votingState=VOTING], PeerInfo [id=follower606, address=pekko://test/user/follower606, votingState=NON_VOTING], PeerInfo [id=leader603, address=pekko://test/user/leader603, votingState=VOTING]] 10:39:57,735 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower605 in 64.74 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:57,736 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Recovery completed in 671.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:57.736] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower606]: org.opendaylight.controller.cluster.common.actor.Monitor@7959209 10:39:57,737 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower606 with journal batch size 1000 10:39:57,737 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: SnapshotOffer called. 10:39:57,737 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Moving last applied index from -1 to -1 10:39:57,737 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower606: Set currentTerm=1, votedFor=null 10:39:57,737 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Updated server config: isVoting: false, peers: [PeerInfo [id=follower605, address=pekko://test/user/follower605, votingState=VOTING], PeerInfo [id=follower604, address=pekko://test/user/follower604, votingState=VOTING], PeerInfo [id=leader603, address=pekko://test/user/leader603, votingState=VOTING]] 10:39:57,737 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower606 in 41.42 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/29/2025 22:39:57.738] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader603]: org.opendaylight.controller.cluster.common.actor.Monitor@39c132d2 10:39:57,738 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Recovery completed in 479.8 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:57,738 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader603 with journal batch size 1000 10:39:57,739 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: SnapshotOffer called. 10:39:57,739 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Moving last applied index from -1 to -1 10:39:57,739 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader603: Set currentTerm=1, votedFor=null 10:39:57,739 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Updated server config: isVoting: true, peers: [PeerInfo [id=follower605, address=pekko://test/user/follower605, votingState=VOTING], PeerInfo [id=follower604, address=pekko://test/user/follower604, votingState=VOTING], PeerInfo [id=follower606, address=pekko://test/user/follower606, votingState=NON_VOTING]] 10:39:57,739 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader603 in 69.30 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:57,739 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Recovery completed in 515.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.31:17355, Removed) 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Follower): Received ElectionTimeout - switching to Candidate 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate): Election: Candidate has following voting peers: [follower605, follower604] 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - leader603: Set currentTerm=2, votedFor=leader603 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate): Starting new election term 2 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate): Sending RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1} to peer follower605 10:39:58,78 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate): Sending RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1} to peer follower604 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Cluster state: [] 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=2, votedFor=null 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Cluster state: [] 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower605: Set currentTerm=2, votedFor=null 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): In requestVote: RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=2, votedFor=leader603 10:39:58,79 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): In requestVote: RequestVote{term=2, candidateId=leader603, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower605: Set currentTerm=2, votedFor=leader603 10:39:58,79 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:58,79 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 10:39:58,81 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 10:39:58,81 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 10:39:58,81 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Election: Leader has following peers: [follower605, follower604, follower606] 10:39:58,82 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower605: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,82 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower604: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,82 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower606: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,82 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 10:39:58,82 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower605: Last sync leader does not match current leader leader603, need to catch up to -1 10:39:58,82 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower604: Last sync leader does not match current leader leader603, need to catch up to -1 10:39:58,82 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 10:39:58,82 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower606: Set currentTerm=2, votedFor=null 10:39:58,82 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower605 updated: matchIndex: -1, nextIndex: 0 10:39:58,82 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower606: Last sync leader does not match current leader leader603, need to catch up to -1 10:39:58,82 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,83 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower604 updated: matchIndex: -1, nextIndex: 0 10:39:58,83 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:58.082] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower604] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@36e08477 [WARN] [01/29/2025 22:39:58.083] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower605] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@584c5939 10:39:58,83 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower606 updated: matchIndex: -1, nextIndex: 0 10:39:58,83 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower606: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:58.083] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower606] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@4d787c8b 10:39:58,92 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:58,92 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 10:39:58,93 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 10:39:58,93 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging starting 10:39:58,93 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Persist data SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:58,93 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:58,93 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 10:39:58,93 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,93 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): sendAppendEntries: 0 is present for follower follower605 10:39:58,93 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Sending AppendEntries to follower follower605: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,94 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,94 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): sendAppendEntries: 0 is present for follower follower604 10:39:58,94 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Sending AppendEntries to follower follower604: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Number of entries to be appended = 1 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Append entry to log zero 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Log size is now 1 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower604, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 10:39:58,94 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower604: Lagging 0 entries behind leader leader603 and reached -1 (of expected -1) 10:39:58,94 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower606: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,94 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): sendAppendEntries: 0 is present for follower follower606 10:39:58,94 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Sending AppendEntries to follower follower606: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,95 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower604 updated: matchIndex: 0, nextIndex: 1 10:39:58,95 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,95 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): Number of entries to be appended = 1 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): Append entry to log zero 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] [WARN] [01/29/2025 22:39:58.095] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower604] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@7579e4ab 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): Log size is now 1 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower606, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 10:39:58,95 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower606: Lagging 0 entries behind leader leader603 and reached -1 (of expected -1) 10:39:58,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower606 updated: matchIndex: 0, nextIndex: 1 10:39:58,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower606: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 10:39:58,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 10:39:58,96 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Setting last applied to 0 10:39:58,96 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Moving last applied index from -1 to 0 10:39:58,96 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Applying state for log index 0 data zero 10:39:58,96 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: applyState called: zero 10:39:58,96 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Persisting ApplyJournalEntries with index=0 10:39:58,96 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] [WARN] [01/29/2025 22:39:58.096] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower606] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6c63ea8c 10:39:58,197 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): sendAppendEntries: 0 is present for follower follower605 10:39:58,197 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Sending AppendEntries to follower follower605: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,197 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): Commit index set to 0 10:39:58,197 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Commit index set to 0 10:39:58,197 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower604: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,197 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower606: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,197 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 10:39:58,197 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 10:39:58,197 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Setting last applied to 0 10:39:58,197 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606 (Follower): Setting last applied to 0 10:39:58,198 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Moving last applied index from -1 to 0 10:39:58,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Moving last applied index from -1 to 0 10:39:58,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Applying state for log index 0 data zero 10:39:58,198 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Applying state for log index 0 data zero 10:39:58,198 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: applyState called: zero 10:39:58,198 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: applyState called: zero 10:39:58,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: Persisting ApplyJournalEntries with index=0 10:39:58,198 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Persisting ApplyJournalEntries with index=0 10:39:58,246 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging ending 10:39:58,246 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: onRequestLeadership RequestLeadership [requestedFollowerId=follower605, replyTo=Actor[pekko://test/user/$c#1614251216]] 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: Initiating leader transfer 10:39:58,246 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: onLeaderTransitioning: LeaderTransitioning [leaderId=leader603] 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorLeadershipTransferCohort - leader603: pauseLeader successfully completed - doing transfer 10:39:58,246 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: onLeaderTransitioning: LeaderTransitioning [leaderId=leader603] 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Attempting to transfer leadership 10:39:58,246 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower606: onLeaderTransitioning: LeaderTransitioning [leaderId=leader603] 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): sendAppendEntries: 0 is present for follower follower605 10:39:58,246 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,246 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 10:39:58,247 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Checking sendAppendEntries for follower follower606: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 10:39:58,247 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower606: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,247 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower604: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,317 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): sendAppendEntries: 0 is present for follower follower605 10:39:58,317 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Sending AppendEntries to follower follower605: AppendEntries{term=2, leaderId=leader603, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 10:39:58,438 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower604: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,438 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower606: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,558 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - leader603: leader transfer aborted 10:39:58,558 PM [test-pekko.actor.default-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - leader603: Failed to transfer leadership in 311.5 ms 10:39:58,558 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603: LeadershipTransfer request from follower605 failed 10:39:58,558 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader603 (Leader): Leadership transfer expired 10:39:58,558 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower604: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,559 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower606: Lagging 0 entries behind leader leader603 and reached 0 (of expected -1) 10:39:58,600 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Lagging ending 10:39:58,601 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower604-notifier607#-1769399010] 10:39:58,611 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower604-collector608#1342516483] 10:39:58,612 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower604] 10:39:58,612 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower605-notifier609#-618045230] 10:39:58,612 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower605-collector610#768488903] 10:39:58,612 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower605] 10:39:58,613 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower606-notifier611#56560334] 10:39:58,613 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower606-collector612#497906289] 10:39:58,613 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower606] 10:39:58,613 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader603-notifier613#-1914792791] 10:39:58,613 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader603-collector614#1959649875] 10:39:58,613 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader603] 10:39:58,614 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$c#1614251216] 10:39:58,615 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithOtherFollowersDown starting 10:39:58,615 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/29/2025 22:39:58.617] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower616]: org.opendaylight.controller.cluster.common.actor.Monitor@2cb7464c 10:39:58,617 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower616 with journal batch size 1000 10:39:58,618 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: SnapshotOffer called. 10:39:58,618 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Moving last applied index from -1 to -1 10:39:58,618 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=1, votedFor=null 10:39:58,618 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Updated server config: isVoting: true, peers: [PeerInfo [id=follower618, address=pekko://test/user/follower618, votingState=NON_VOTING], PeerInfo [id=follower617, address=pekko://test/user/follower617, votingState=VOTING], PeerInfo [id=leader615, address=pekko://test/user/leader615, votingState=VOTING]] 10:39:58,618 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower616 in 88.29 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:58,619 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Recovery completed in 1.002 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:58.620] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower617]: org.opendaylight.controller.cluster.common.actor.Monitor@617a6f5a 10:39:58,620 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower617 with journal batch size 1000 10:39:58,621 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: SnapshotOffer called. 10:39:58,621 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Moving last applied index from -1 to -1 10:39:58,621 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=1, votedFor=null 10:39:58,621 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Updated server config: isVoting: true, peers: [PeerInfo [id=follower616, address=pekko://test/user/follower616, votingState=VOTING], PeerInfo [id=follower618, address=pekko://test/user/follower618, votingState=NON_VOTING], PeerInfo [id=leader615, address=pekko://test/user/leader615, votingState=VOTING]] 10:39:58,621 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower617 in 110.4 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:58,622 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Recovery completed in 1.052 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:58.622] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower618]: org.opendaylight.controller.cluster.common.actor.Monitor@738e170 10:39:58,622 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower618 with journal batch size 1000 10:39:58,623 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower618: SnapshotOffer called. 10:39:58,623 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower618: Moving last applied index from -1 to -1 10:39:58,623 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower618: Set currentTerm=1, votedFor=null 10:39:58,623 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower618: Updated server config: isVoting: false, peers: [PeerInfo [id=follower616, address=pekko://test/user/follower616, votingState=VOTING], PeerInfo [id=follower617, address=pekko://test/user/follower617, votingState=VOTING], PeerInfo [id=leader615, address=pekko://test/user/leader615, votingState=VOTING]] 10:39:58,623 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower618 in 38.84 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:58,623 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower618: Recovery completed in 468.0 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [01/29/2025 22:39:58.624] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader615]: org.opendaylight.controller.cluster.common.actor.Monitor@e21f5bf 10:39:58,624 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader615 with journal batch size 1000 10:39:58,624 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: SnapshotOffer called. 10:39:58,624 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: Moving last applied index from -1 to -1 10:39:58,624 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PersistenceTermInfoStore - leader615: Set currentTerm=1, votedFor=null 10:39:58,624 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: Updated server config: isVoting: true, peers: [PeerInfo [id=follower616, address=pekko://test/user/follower616, votingState=VOTING], PeerInfo [id=follower618, address=pekko://test/user/follower618, votingState=NON_VOTING], PeerInfo [id=follower617, address=pekko://test/user/follower617, votingState=VOTING]] 10:39:58,624 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader615 in 42.51 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 10:39:58,625 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: Recovery completed in 481.5 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 10:39:58,957 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:58,957 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.31:17355, Removed) 10:39:58,957 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Follower): Received ElectionTimeout - switching to Candidate 10:39:58,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate): Election: Candidate has following voting peers: [follower616, follower617] 10:39:58,958 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PersistenceTermInfoStore - leader615: Set currentTerm=2, votedFor=leader615 10:39:58,958 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate): Starting new election term 2 10:39:58,959 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate): Sending RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1} to peer follower616 10:39:58,959 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate): Sending RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1} to peer follower617 10:39:58,959 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:58,959 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 10:39:58,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:58,959 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Cluster state: [] 10:39:58,959 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:58,960 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): Cluster state: [] 10:39:58,960 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=2, votedFor=null 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=2, votedFor=null 10:39:58,960 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): In requestVote: RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:58,960 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=2, votedFor=leader615 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): In requestVote: RequestVote{term=2, candidateId=leader615, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:58,960 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=2, votedFor=leader615 10:39:58,960 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Election: Leader has following peers: [follower616, follower618, follower617] 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower616: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower618: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower617: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,961 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 10:39:58,961 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower618 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader615, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 10:39:58,962 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower618: Set currentTerm=2, votedFor=null 10:39:58,962 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower617: Last sync leader does not match current leader leader615, need to catch up to -1 10:39:58,962 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower618: Last sync leader does not match current leader leader615, need to catch up to -1 10:39:58,962 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower617 updated: matchIndex: -1, nextIndex: 0 10:39:58,962 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower617: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:58,962 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower618 updated: matchIndex: -1, nextIndex: 0 10:39:58,962 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower618: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:58.962] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower617] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@446897ed 10:39:58,962 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower616: Last sync leader does not match current leader leader615, need to catch up to -1 10:39:58,962 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower616 updated: matchIndex: -1, nextIndex: 0 10:39:58,962 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower616: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:58.962] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower616] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@100519e5 [WARN] [01/29/2025 22:39:58.962] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower618] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@76438011 10:39:58,978 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 10:39:58,978 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 10:39:58,978 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 10:39:58,978 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower616] 10:39:58,990 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower618] 10:39:59,0 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 10:39:59,0 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 10:39:59,0 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: onRequestLeadership RequestLeadership [requestedFollowerId=follower617, replyTo=Actor[pekko://test/user/$d#1383950577]] 10:39:59,0 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: Initiating leader transfer [WARN] [01/29/2025 22:39:59.001] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower616] received dead letter from TestActor[pekko://test/user/leader615]: LeaderTransitioning [leaderId=leader615] 10:39:59,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorLeadershipTransferCohort - leader615: pauseLeader successfully completed - doing transfer 10:39:59,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Attempting to transfer leadership [WARN] [01/29/2025 22:39:59.001] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower618] received dead letter from TestActor[pekko://test/user/leader615]: LeaderTransitioning [leaderId=leader615] 10:39:59,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: onLeaderTransitioning: LeaderTransitioning [leaderId=leader615] 10:39:59,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower616: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:59,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower618: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:59.001] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower616] received dead letter from TestActor[pekko://test/user/leader615]: AppendEntries{term=2, leaderId=leader615, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:59,1 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower617: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:59.001] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower618] received dead letter from TestActor[pekko://test/user/leader615]: AppendEntries{term=2, leaderId=leader615, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:59,1 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower617: Lagging 0 entries behind leader leader615 and reached -1 (of expected -1) [WARN] [01/29/2025 22:39:59.002] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower617] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@1b758fc3 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): tryToCompleteLeadershipTransfer: followerId: follower617, matchIndex: -1, lastIndex: -1, isVoting: true 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Follower's log matches - sending ElectionTimeout 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower616: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower618: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Checking sendAppendEntries for follower follower617: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:59.002] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower616] received dead letter from TestActor[pekko://test/user/leader615]: AppendEntries{term=2, leaderId=leader615, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [01/29/2025 22:39:59.002] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower618] received dead letter from TestActor[pekko://test/user/leader615]: AppendEntries{term=2, leaderId=leader615, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 10:39:59,2 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower617: Lagging 0 entries behind leader leader615 and reached -1 (of expected -1) 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Leader transfer complete 10:39:59,2 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorLeadershipTransferCohort - leader615: leader transfer complete - waiting for new leader 10:39:59,2 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 10:39:59,2 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): Received TimeoutNow - switching to Candidate 10:39:59,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate): Election: Candidate has following voting peers: [follower616, leader615] 10:39:59,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=3, votedFor=follower617 10:39:59,3 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate): Starting new election term 3 10:39:59,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate): Sending RequestVote{term=3, candidateId=follower617, lastLogIndex=-1, lastLogTerm=-1} to peer follower616 10:39:59,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate): Sending RequestVote{term=3, candidateId=follower617, lastLogIndex=-1, lastLogTerm=-1} to peer leader615 10:39:59,3 PM [test-pekko.actor.default-dispatcher-14] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower) :- Switching from behavior Follower to Candidate, election term: 3 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Cluster state: [] 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=follower617, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Term 3 in "RequestVote{term=3, candidateId=follower617, lastLogIndex=-1, lastLogTerm=-1}" message is greater than leader's term 2 - switching to Follower 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader615: Set currentTerm=3, votedFor=null 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): Leadership transfer in progress - processing RequestVote 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): In requestVote: RequestVote{term=3, candidateId=follower617, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader615: Set currentTerm=3, votedFor=follower617 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 10:39:59,3 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader615 (Leader) :- Switching from behavior Leader to Follower, election term: 3 [WARN] [01/29/2025 22:39:59.004] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower616] received dead letter from TestActor[pekko://test/user/follower617]: RequestVote{term=3, candidateId=follower617, lastLogIndex=-1, lastLogTerm=-1} 10:39:59,4 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 10:39:59,4 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 10:39:59,4 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Leader): Election: Leader has following peers: [follower616, follower618, leader615] 10:39:59,4 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Leader): Checking sendAppendEntries for follower follower616: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:59,4 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Leader): Checking sendAppendEntries for follower follower618: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:59.004] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower616] received dead letter from TestActor[pekko://test/user/follower617]: AppendEntries{term=3, leaderId=follower617, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 10:39:59,5 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Leader): Checking sendAppendEntries for follower leader615: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/29/2025 22:39:59.004] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower618] received dead letter from TestActor[pekko://test/user/follower617]: AppendEntries{term=3, leaderId=follower617, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} 10:39:59,5 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 10:39:59,5 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - leader615: Last sync leader does not match current leader follower617, need to catch up to -1 10:39:59,5 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - leader615: leader changed to follower617 10:39:59,5 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Leader): handleAppendEntriesReply - FollowerLogInformation for leader615 updated: matchIndex: -1, nextIndex: 0 10:39:59,5 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Leader): Checking sendAppendEntries for follower leader615: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 10:39:59,5 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorLeadershipTransferCohort - leader615: Successfully transferred leadership to follower617 in 4.482 ms 10:39:59,5 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader615: Leadership transferred successfully to follower617 [WARN] [01/29/2025 22:39:59.005] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader615] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@2a923ac1 10:39:59,52 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithOtherFollowersDown ending 10:39:59,52 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower616-notifier619#1620918599] 10:39:59,63 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower616-collector620#-1394767438] 10:39:59,63 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower617-notifier621#963569701] 10:39:59,63 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower617-collector622#-511675745] 10:39:59,63 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower617] 10:39:59,64 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower618-notifier623#1787269545] 10:39:59,64 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower618-collector624#373292792] 10:39:59,64 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader615-notifier625#1507783434] 10:39:59,64 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader615-collector626#-1196372678] 10:39:59,64 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader615] 10:39:59,64 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$d#1383950577] [INFO] [01/29/2025 22:39:59.065] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 10:39:59,65 PM [test-pekko.actor.default-dispatcher-16] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [WARN] [01/29/2025 22:39:59.067] [test-pekko.actor.default-dispatcher-16] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.170.31:17355) [INFO] [01/29/2025 22:39:59.067] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - Exiting completed 10:39:59,67 PM [test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - Exiting completed 10:39:59,67 PM [test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - Shutting down... [INFO] [01/29/2025 22:39:59.067] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - Shutting down... [INFO] [01/29/2025 22:39:59.067] [test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.31:17355] - Successfully shut down 10:39:59,67 PM [test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.31:17355] - Successfully shut down 10:39:59,69 PM [test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 10:39:59,69 PM [test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [INFO] [01/29/2025 22:39:59.069] [test-pekko.remote.default-remote-dispatcher-10] [pekko://test@10.30.170.31:17355/system/remoting-terminator] Shutting down remote daemon. [INFO] [01/29/2025 22:39:59.069] [test-pekko.remote.default-remote-dispatcher-10] [pekko://test@10.30.170.31:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports. 10:39:59,72 PM [test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [INFO] [01/29/2025 22:39:59.072] [test-pekko.remote.default-remote-dispatcher-10] [pekko://test@10.30.170.31:17355/system/remoting-terminator] Remoting shut down. [DEBUG] [01/29/2025 22:39:59.072] [test-pekko.actor.internal-dispatcher-4] [EventStream] shutting down: StandardOutLogger