[DEBUG] [01/31/2025 13:35:40.915] [main] [EventStream(pekko://test)] logger log1-TestEventListener started 01:35:40,915 PM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started [DEBUG] [01/31/2025 13:35:40.915] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started [DEBUG] [01/31/2025 13:35:40.915] [main] [EventStream(pekko://test)] Default Loggers started [INFO] [01/31/2025 13:35:40.928] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.157:17355] with UID [-780983842280053697] [INFO] [01/31/2025 13:35:40.929] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - Starting up, Pekko version [1.0.3] ... 01:35:40,929 PM [test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.157:17355] with UID [-780983842280053697] 01:35:40,929 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - Starting up, Pekko version [1.0.3] ... 01:35:40,937 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 01:35:40,937 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - Started up successfully 01:35:40,937 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 01:35:40,937 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining [INFO] [01/31/2025 13:35:40.933] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - Registered cluster JMX MBean [pekko:type=Cluster] [INFO] [01/31/2025 13:35:40.933] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - Started up successfully [INFO] [01/31/2025 13:35:40.934] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing [INFO] [01/31/2025 13:35:40.934] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 01:35:40,942 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferAborted starting 01:35:40,942 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/31/2025 13:35:40.946] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower550]: org.opendaylight.controller.cluster.common.actor.Monitor@1ab3b4f3 01:35:40,947 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower550] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/31/2025 13:35:40.946] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower550] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:40,949 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower550 with journal batch size 1000 01:35:40,951 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower551 with journal batch size 1000 [WARN] [01/31/2025 13:35:40.951] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower551]: org.opendaylight.controller.cluster.common.actor.Monitor@7f13089b [INFO] [pekkoDeadLetter][01/31/2025 13:35:40.951] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower551] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:40,951 PM [test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower551] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:40,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: SnapshotOffer called. 01:35:40,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Moving last applied index from -1 to -1 01:35:40,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower550: Set currentTerm=1, votedFor=null 01:35:40,953 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Updated server config: isVoting: true, peers: [PeerInfo [id=follower552, address=pekko://test/user/follower552, votingState=NON_VOTING], PeerInfo [id=leader549, address=pekko://test/user/leader549, votingState=VOTING], PeerInfo [id=follower551, address=pekko://test/user/follower551, votingState=VOTING]] 01:35:40,953 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower550 in 78.91 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:40,953 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Recovery completed in 687.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:40,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower550: Set currentTerm=1, votedFor=null 01:35:40,954 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Local TermInfo store seeded with TermInfo{term=1} 01:35:40,954 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Snapshot capture initiated after recovery due to migrated messages 01:35:40,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:40,954 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:40,954 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: lastSequenceNumber prior to capture: 0 01:35:40,954 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: SnapshotOffer called. 01:35:40,954 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Moving last applied index from -1 to -1 01:35:40,954 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=1, votedFor=null 01:35:40,954 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Updated server config: isVoting: true, peers: [PeerInfo [id=follower552, address=pekko://test/user/follower552, votingState=NON_VOTING], PeerInfo [id=leader549, address=pekko://test/user/leader549, votingState=VOTING], PeerInfo [id=follower550, address=pekko://test/user/follower550, votingState=VOTING]] 01:35:40,954 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower551 in 66.16 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:40,955 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: CaptureSnapshotReply received by actor 01:35:40,955 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower552, isVoting=false], ServerInfo[peerId=leader549, isVoting=true], ServerInfo[peerId=follower551, isVoting=true], ServerInfo[peerId=follower550, isVoting=true]]]] 01:35:40,955 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Recovery completed in 1.776 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=1, votedFor=null 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Local TermInfo store seeded with TermInfo{term=1} 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Snapshot capture initiated after recovery due to migrated messages 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:40,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: lastSequenceNumber prior to capture: 0 01:35:40,957 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:40,957 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: Snapshot success - sequence number: -1 01:35:40,957 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: SnapshotComplete received 01:35:40,957 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: handled a snapshot response 01:35:40,958 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: CaptureSnapshotReply received by actor 01:35:40,958 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower552, isVoting=false], ServerInfo[peerId=leader549, isVoting=true], ServerInfo[peerId=follower550, isVoting=true], ServerInfo[peerId=follower551, isVoting=true]]]] 01:35:40,958 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:40,958 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:40,958 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: Snapshot success - sequence number: -1 01:35:40,958 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: handled a journal response 01:35:40,958 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: SnapshotComplete received 01:35:40,958 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: handled a snapshot response 01:35:40,959 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: handled a journal response [WARN] [01/31/2025 13:35:41.053] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower552]: org.opendaylight.controller.cluster.common.actor.Monitor@71c18fb5 [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.053] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower552] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,53 PM [test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower552] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,53 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower552 with journal batch size 1000 01:35:41,54 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: SnapshotOffer called. 01:35:41,54 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Moving last applied index from -1 to -1 01:35:41,54 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower552: Set currentTerm=1, votedFor=null 01:35:41,54 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Updated server config: isVoting: false, peers: [PeerInfo [id=leader549, address=pekko://test/user/leader549, votingState=VOTING], PeerInfo [id=follower551, address=pekko://test/user/follower551, votingState=VOTING], PeerInfo [id=follower550, address=pekko://test/user/follower550, votingState=VOTING]] 01:35:41,54 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower552 in 95.08 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/31/2025 13:35:41.055] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader549]: org.opendaylight.controller.cluster.common.actor.Monitor@2cab40a9 [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.055] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader549] to Actor[pekko://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,55 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader549] to Actor[pekko://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader549 with journal batch size 1000 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: SnapshotOffer called. 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Moving last applied index from -1 to -1 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - leader549: Set currentTerm=1, votedFor=null 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Updated server config: isVoting: true, peers: [PeerInfo [id=follower552, address=pekko://test/user/follower552, votingState=NON_VOTING], PeerInfo [id=follower551, address=pekko://test/user/follower551, votingState=VOTING], PeerInfo [id=follower550, address=pekko://test/user/follower550, votingState=VOTING]] 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader549 in 65.29 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Recovery completed in 2.203 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower552: Set currentTerm=1, votedFor=null 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Local TermInfo store seeded with TermInfo{term=1} 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Snapshot capture initiated after recovery due to migrated messages 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:41,56 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:41,57 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: lastSequenceNumber prior to capture: 0 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Recovery completed in 805.0 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader549: Set currentTerm=1, votedFor=null 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Local TermInfo store seeded with TermInfo{term=1} 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Snapshot capture initiated after recovery due to migrated messages 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:41,57 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: lastSequenceNumber prior to capture: 0 01:35:41,59 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: CaptureSnapshotReply received by actor 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: CaptureSnapshotReply received by actor 01:35:41,60 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader549, isVoting=true], ServerInfo[peerId=follower551, isVoting=true], ServerInfo[peerId=follower550, isVoting=true], ServerInfo[peerId=follower552, isVoting=false]]]] 01:35:41,60 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:41,60 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:41,60 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: Snapshot success - sequence number: -1 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower552, isVoting=false], ServerInfo[peerId=follower551, isVoting=true], ServerInfo[peerId=follower550, isVoting=true], ServerInfo[peerId=leader549, isVoting=true]]]] 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Snapshot success - sequence number: -1 01:35:41,60 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: SnapshotComplete received 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: SnapshotComplete received 01:35:41,60 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: handled a snapshot response 01:35:41,60 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: handled a snapshot response 01:35:41,62 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: handled a journal response 01:35:41,62 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: handled a journal response 01:35:41,452 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:41,452 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.157:17355, Removed) 01:35:41,452 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Follower): Received ElectionTimeout - switching to Candidate 01:35:41,452 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate): Election: Candidate has following voting peers: [follower551, follower550] 01:35:41,452 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader549: Set currentTerm=2, votedFor=leader549 01:35:41,453 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate): Starting new election term 2 01:35:41,453 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate): Sending RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1} to peer follower551 01:35:41,453 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate): Sending RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1} to peer follower550 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:41,453 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Cluster state: [] 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=2, votedFor=null 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): In requestVote: RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550 (Follower): Cluster state: [] 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower551: Set currentTerm=2, votedFor=leader549 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower550 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower550: Set currentTerm=2, votedFor=null 01:35:41,453 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550 (Follower): In requestVote: RequestVote{term=2, candidateId=leader549, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower550: Set currentTerm=2, votedFor=leader549 01:35:41,453 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Election: Leader has following peers: [follower552, follower551, follower550] 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower552: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower551: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower550: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower552 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader549, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 01:35:41,455 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower552: Set currentTerm=2, votedFor=null 01:35:41,456 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower552: Last sync leader does not match current leader leader549, need to catch up to -1 01:35:41,456 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower551: Last sync leader does not match current leader leader549, need to catch up to -1 01:35:41,456 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower552 updated: matchIndex: -1, nextIndex: 0 01:35:41,456 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower552: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,456 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower551 updated: matchIndex: -1, nextIndex: 0 01:35:41,456 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower551: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,456 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower550: Last sync leader does not match current leader leader549, need to catch up to -1 01:35:41,456 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower550 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/31/2025 13:35:41.456] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower550] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6465c52d 01:35:41,456 PM [test-pekko.actor.default-dispatcher-5] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower550] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.456] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower550] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower550] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,457 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower550: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:41.457] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower551] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@1ab2fc93 [WARN] [01/31/2025 13:35:41.457] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower552] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@10e0bdb6 01:35:41,457 PM [test-pekko.actor.default-dispatcher-6] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,457 PM [test-pekko.actor.default-dispatcher-6] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.457] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower551] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.457] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower552] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,462 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:41,462 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader549 starting 01:35:41,463 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: onShutDown 01:35:41,463 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: Initiating leader transfer 01:35:41,464 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower552: onLeaderTransitioning: LeaderTransitioning [leaderId=leader549] 01:35:41,464 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower550: onLeaderTransitioning: LeaderTransitioning [leaderId=leader549] 01:35:41,464 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower551: onLeaderTransitioning: LeaderTransitioning [leaderId=leader549] 01:35:41,464 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - leader549: pauseLeader successfully completed - doing transfer 01:35:41,464 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Attempting to transfer leadership 01:35:41,464 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower552: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,464 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:41.464] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower552] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6ac5e1d1 [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.464] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower552] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,464 PM [test-pekko.actor.default-dispatcher-12] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower552] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,465 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower551: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,465 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:41.465] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower551] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@4f33fa1c [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.465] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower551] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,465 PM [test-pekko.actor.default-dispatcher-5] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower551] was unhandled. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,465 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Checking sendAppendEntries for follower follower550: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:41,465 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower550: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,465 PM [test-pekko.actor.default-dispatcher-6] [INFO] TestActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower550] was unhandled. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [WARN] [01/31/2025 13:35:41.466] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower550] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@698f5859 [INFO] [pekkoDeadLetter][01/31/2025 13:35:41.465] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower550] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower550] was unhandled. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 01:35:41,573 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower550: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,573 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,573 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,693 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,693 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower550: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,693 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,812 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - leader549: leader transfer aborted 01:35:41,812 PM [test-pekko.actor.default-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - leader549: Failed to transfer leadership in 349.0 ms 01:35:41,813 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549: leader transfer failed - sending PoisonPill 01:35:41,813 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader549 (Leader): Leadership transfer expired 01:35:41,813 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower552: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,813 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower550: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower551: Lagging 0 entries behind leader leader549 and reached -1 (of expected -1) 01:35:41,815 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader549 ending 01:35:41,816 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferOnShutDown ending 01:35:41,816 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower550-notifier553#1222599251] 01:35:41,828 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower550-collector554#1099904063] 01:35:41,828 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower550] 01:35:41,828 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower551-notifier555#2012421948] 01:35:41,829 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower551-collector556#1034880005] 01:35:41,829 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower551] 01:35:41,829 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower552-notifier557#1556051812] 01:35:41,829 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower552-collector558#-401967868] 01:35:41,829 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower552] 01:35:41,829 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader549-notifier559#-1295965195] 01:35:41,830 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader549-collector560#-1548706528] 01:35:41,830 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader549] 01:35:41,831 PM [main] [INFO] LeadershipTransferIntegrationTest - testSuccessfulRequestLeadershipTransferToFollower2 starting 01:35:41,831 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/31/2025 13:35:41.833] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower562]: org.opendaylight.controller.cluster.common.actor.Monitor@2b4049cb 01:35:41,834 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower562 with journal batch size 1000 01:35:41,834 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: SnapshotOffer called. 01:35:41,834 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Moving last applied index from -1 to -1 01:35:41,834 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower562: Set currentTerm=1, votedFor=null 01:35:41,834 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Updated server config: isVoting: true, peers: [PeerInfo [id=follower564, address=pekko://test/user/follower564, votingState=NON_VOTING], PeerInfo [id=follower563, address=pekko://test/user/follower563, votingState=VOTING], PeerInfo [id=leader561, address=pekko://test/user/leader561, votingState=VOTING]] 01:35:41,835 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower562 in 94.97 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Recovery completed in 793.0 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower562: Set currentTerm=1, votedFor=null 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Local TermInfo store seeded with TermInfo{term=1} 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Snapshot capture initiated after recovery due to migrated messages 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:41,835 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: lastSequenceNumber prior to capture: 0 [WARN] [01/31/2025 13:35:41.836] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower563]: org.opendaylight.controller.cluster.common.actor.Monitor@7dcce0e 01:35:41,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: CaptureSnapshotReply received by actor 01:35:41,837 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower564, isVoting=false], ServerInfo[peerId=follower563, isVoting=true], ServerInfo[peerId=leader561, isVoting=true], ServerInfo[peerId=follower562, isVoting=true]]]] 01:35:41,837 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:41,837 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:41,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: Snapshot success - sequence number: -1 01:35:41,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: SnapshotComplete received 01:35:41,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: handled a snapshot response 01:35:41,837 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: handled a journal response 01:35:41,839 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower563 with journal batch size 1000 01:35:41,839 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: SnapshotOffer called. 01:35:41,839 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Moving last applied index from -1 to -1 01:35:41,839 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=1, votedFor=null 01:35:41,839 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Updated server config: isVoting: true, peers: [PeerInfo [id=follower564, address=pekko://test/user/follower564, votingState=NON_VOTING], PeerInfo [id=leader561, address=pekko://test/user/leader561, votingState=VOTING], PeerInfo [id=follower562, address=pekko://test/user/follower562, votingState=VOTING]] 01:35:41,839 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower563 in 46.98 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/31/2025 13:35:41.840] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower564]: org.opendaylight.controller.cluster.common.actor.Monitor@3a7072b4 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Recovery completed in 1.092 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=1, votedFor=null 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Local TermInfo store seeded with TermInfo{term=1} 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Snapshot capture initiated after recovery due to migrated messages 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:41,840 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: lastSequenceNumber prior to capture: 0 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: CaptureSnapshotReply received by actor 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower564, isVoting=false], ServerInfo[peerId=leader561, isVoting=true], ServerInfo[peerId=follower562, isVoting=true], ServerInfo[peerId=follower563, isVoting=true]]]] 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: Snapshot success - sequence number: -1 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: SnapshotComplete received 01:35:41,841 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: handled a snapshot response 01:35:41,841 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: handled a journal response 01:35:41,841 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower564 with journal batch size 1000 01:35:41,842 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: SnapshotOffer called. 01:35:41,842 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Moving last applied index from -1 to -1 01:35:41,842 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=1, votedFor=null 01:35:41,842 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Updated server config: isVoting: false, peers: [PeerInfo [id=follower563, address=pekko://test/user/follower563, votingState=VOTING], PeerInfo [id=leader561, address=pekko://test/user/leader561, votingState=VOTING], PeerInfo [id=follower562, address=pekko://test/user/follower562, votingState=VOTING]] 01:35:41,842 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower564 in 42.82 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:41,842 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Recovery completed in 585.5 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:41,842 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=1, votedFor=null [WARN] [01/31/2025 13:35:41.843] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader561]: org.opendaylight.controller.cluster.common.actor.Monitor@651dc5b6 01:35:41,843 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader561 with journal batch size 1000 01:35:41,843 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: SnapshotOffer called. 01:35:41,843 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Moving last applied index from -1 to -1 01:35:41,843 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - leader561: Set currentTerm=1, votedFor=null 01:35:41,843 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Updated server config: isVoting: true, peers: [PeerInfo [id=follower564, address=pekko://test/user/follower564, votingState=NON_VOTING], PeerInfo [id=follower563, address=pekko://test/user/follower563, votingState=VOTING], PeerInfo [id=follower562, address=pekko://test/user/follower562, votingState=VOTING]] 01:35:41,843 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader561 in 40.61 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:41,843 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Local TermInfo store seeded with TermInfo{term=1} 01:35:41,843 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Snapshot capture initiated after recovery due to migrated messages 01:35:41,843 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:41,843 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:41,844 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: lastSequenceNumber prior to capture: 0 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Recovery completed in 428.2 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader561: Set currentTerm=1, votedFor=null 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Local TermInfo store seeded with TermInfo{term=1} 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Snapshot capture initiated after recovery due to migrated messages 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:41,844 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: lastSequenceNumber prior to capture: 0 01:35:41,844 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: CaptureSnapshotReply received by actor 01:35:41,844 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower563, isVoting=true], ServerInfo[peerId=leader561, isVoting=true], ServerInfo[peerId=follower562, isVoting=true], ServerInfo[peerId=follower564, isVoting=false]]]] 01:35:41,844 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:41,845 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: CaptureSnapshotReply received by actor 01:35:41,845 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:41,845 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: Snapshot success - sequence number: -1 01:35:41,845 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower564, isVoting=false], ServerInfo[peerId=follower563, isVoting=true], ServerInfo[peerId=follower562, isVoting=true], ServerInfo[peerId=leader561, isVoting=true]]]] 01:35:41,845 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: SnapshotComplete received 01:35:41,845 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:41,845 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: handled a journal response 01:35:41,845 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:41,845 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Snapshot success - sequence number: -1 01:35:41,845 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: handled a snapshot response 01:35:41,845 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: SnapshotComplete received 01:35:41,845 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: handled a journal response 01:35:41,845 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: handled a snapshot response 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.157:17355, Removed) 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Follower): Received ElectionTimeout - switching to Candidate 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate): Election: Candidate has following voting peers: [follower563, follower562] 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader561: Set currentTerm=2, votedFor=leader561 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate): Starting new election term 2 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate): Sending RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1} to peer follower563 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate): Sending RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1} to peer follower562 01:35:42,252 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:35:42,252 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Cluster state: [] 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Cluster state: [] 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=2, votedFor=null 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower562: Set currentTerm=2, votedFor=null 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): In requestVote: RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): In requestVote: RequestVote{term=2, candidateId=leader561, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=2, votedFor=leader561 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower562: Set currentTerm=2, votedFor=leader561 01:35:42,253 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:42,253 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:42,254 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Election: Leader has following peers: [follower564, follower563, follower562] 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower564: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower563: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower562: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 01:35:42,255 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower563: Last sync leader does not match current leader leader561, need to catch up to -1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower562: Last sync leader does not match current leader leader561, need to catch up to -1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower563 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/31/2025 13:35:42.255] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower563] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3dfe04c8 [WARN] [01/31/2025 13:35:42.255] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower562] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@45718c86 01:35:42,255 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,255 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader561, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 01:35:42,256 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=2, votedFor=null 01:35:42,256 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower562 updated: matchIndex: -1, nextIndex: 0 01:35:42,256 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower562: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,256 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower564: Last sync leader does not match current leader leader561, need to catch up to -1 01:35:42,256 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower564 updated: matchIndex: -1, nextIndex: 0 01:35:42,256 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:42.256] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower564] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3e68b3a1 01:35:42,297 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:42,297 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 01:35:42,298 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 01:35:42,298 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 01:35:42,298 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 01:35:42,299 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: onRequestLeadership RequestLeadership [requestedFollowerId=follower563, replyTo=Actor[pekko://test/user/$a#1777893780]] 01:35:42,300 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Initiating leader transfer 01:35:42,300 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower564: onLeaderTransitioning: LeaderTransitioning [leaderId=leader561] 01:35:42,300 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563: onLeaderTransitioning: LeaderTransitioning [leaderId=leader561] 01:35:42,301 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562: onLeaderTransitioning: LeaderTransitioning [leaderId=leader561] 01:35:42,301 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - leader561: pauseLeader successfully completed - doing transfer 01:35:42,301 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Attempting to transfer leadership 01:35:42,301 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,301 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower564: Lagging 0 entries behind leader leader561 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:42.301] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower564] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@ee16180 01:35:42,301 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,301 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower563: Lagging 0 entries behind leader leader561 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:42.302] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower563] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3df2577b 01:35:42,302 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower562: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,302 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower562: Lagging 0 entries behind leader leader561 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:42.302] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower562] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@370a3c0b 01:35:42,302 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): tryToCompleteLeadershipTransfer: followerId: follower563, matchIndex: -1, lastIndex: -1, isVoting: true 01:35:42,302 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Follower's log matches - sending ElectionTimeout 01:35:42,302 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,302 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower563: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,302 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower564: Lagging 0 entries behind leader leader561 and reached -1 (of expected -1) 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower563: Lagging 0 entries behind leader leader561 and reached -1 (of expected -1) 01:35:42,302 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Checking sendAppendEntries for follower follower562: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower): Received TimeoutNow - switching to Candidate 01:35:42,302 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower562: Lagging 0 entries behind leader leader561 and reached -1 (of expected -1) 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate): Election: Candidate has following voting peers: [leader561, follower562] 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower563: Set currentTerm=3, votedFor=follower563 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate): Starting new election term 3 01:35:42,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate): Sending RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1} to peer leader561 01:35:42,303 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate): Sending RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1} to peer follower562 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:42,303 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Follower) :- Switching from behavior Follower to Candidate, election term: 3 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Cluster state: [] 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): Term 3 in "RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 2 - updating term 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower562: Set currentTerm=3, votedFor=null 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): In requestVote: RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower562: Set currentTerm=3, votedFor=follower563 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower562 (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 01:35:42,303 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Leader transfer complete 01:35:42,303 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - leader561: leader transfer complete - waiting for new leader 01:35:42,303 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:42,303 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Cluster state: [] 01:35:42,303 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:42,303 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Term 3 in "RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1}" message is greater than leader's term 2 - switching to Follower 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 01:35:42,303 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader561: Set currentTerm=3, votedFor=null 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): Leadership transfer in progress - processing RequestVote 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Election: Leader has following peers: [follower564, leader561, follower562] 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): In requestVote: RequestVote{term=3, candidateId=follower563, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Checking sendAppendEntries for follower follower564: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - leader561: Set currentTerm=3, votedFor=follower563 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Checking sendAppendEntries for follower leader561: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Checking sendAppendEntries for follower follower562: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader561 (Leader) :- Switching from behavior Leader to Follower, election term: 3 01:35:42,304 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower564 (Follower): Term 3 in "AppendEntries{term=3, leaderId=follower563, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 01:35:42,304 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower562: Last sync leader does not match current leader follower563, need to catch up to -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower564: Set currentTerm=3, votedFor=null 01:35:42,304 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower564: Last sync leader does not match current leader follower563, need to catch up to -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower562 updated: matchIndex: -1, nextIndex: 0 01:35:42,304 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Checking sendAppendEntries for follower follower562: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower564 updated: matchIndex: -1, nextIndex: 0 01:35:42,304 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Checking sendAppendEntries for follower follower564: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:42.304] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower562] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@5d6e7c3b 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - leader561: Last sync leader does not match current leader follower563, need to catch up to -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): handleAppendEntriesReply - FollowerLogInformation for leader561 updated: matchIndex: -1, nextIndex: 0 01:35:42,304 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower563 (Leader): Checking sendAppendEntries for follower leader561: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorLeadershipTransferCohort - leader561: leader changed to follower563 01:35:42,304 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorLeadershipTransferCohort - leader561: Successfully transferred leadership to follower563 in 4.116 ms 01:35:42,305 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader561: Leadership transferred successfully to follower563 [WARN] [01/31/2025 13:35:42.305] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/leader561] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3a3daa2c [WARN] [01/31/2025 13:35:42.305] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower564] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@3fd5218a 01:35:42,350 PM [main] [INFO] LeadershipTransferIntegrationTest - testSuccessfulRequestLeadershipTransferToFollower2 ending 01:35:42,350 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower562-notifier565#2097140254] 01:35:42,361 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower562-collector566#137615695] 01:35:42,361 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower562] 01:35:42,361 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower563-notifier567#-1943765941] 01:35:42,362 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower563-collector568#-1793131560] 01:35:42,362 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower563] 01:35:42,362 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower564-notifier569#1110987873] 01:35:42,362 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower564-collector570#-370121269] 01:35:42,362 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower564] 01:35:42,362 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader561-notifier571#-870201239] 01:35:42,363 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader561-collector572#-1399256854] 01:35:42,363 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader561] 01:35:42,363 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$a#1777893780] 01:35:42,364 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Shutdown starting 01:35:42,364 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:42,368 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower574 with journal batch size 1000 [WARN] [01/31/2025 13:35:42.368] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower574]: org.opendaylight.controller.cluster.common.actor.Monitor@2d3fc0f2 01:35:42,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: SnapshotOffer called. 01:35:42,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Moving last applied index from -1 to -1 01:35:42,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower574: Set currentTerm=1, votedFor=null 01:35:42,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Updated server config: isVoting: true, peers: [PeerInfo [id=leader573, address=pekko://test/user/leader573, votingState=VOTING], PeerInfo [id=follower575, address=pekko://test/user/follower575, votingState=VOTING], PeerInfo [id=follower576, address=pekko://test/user/follower576, votingState=NON_VOTING]] 01:35:42,369 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower574 in 169.4 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Recovery completed in 921.2 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower574: Set currentTerm=1, votedFor=null 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Local TermInfo store seeded with TermInfo{term=1} 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Snapshot capture initiated after recovery due to migrated messages 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:42,370 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: lastSequenceNumber prior to capture: 0 01:35:42,370 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: CaptureSnapshotReply received by actor 01:35:42,370 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader573, isVoting=true], ServerInfo[peerId=follower575, isVoting=true], ServerInfo[peerId=follower576, isVoting=false], ServerInfo[peerId=follower574, isVoting=true]]]] 01:35:42,370 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:42,371 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:42,371 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: Snapshot success - sequence number: -1 01:35:42,371 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: SnapshotComplete received 01:35:42,371 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: handled a journal response 01:35:42,371 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: handled a snapshot response [WARN] [01/31/2025 13:35:42.469] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower575]: org.opendaylight.controller.cluster.common.actor.Monitor@f9096f0 01:35:42,470 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower575 with journal batch size 1000 01:35:42,471 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: SnapshotOffer called. 01:35:42,471 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Moving last applied index from -1 to -1 01:35:42,471 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=1, votedFor=null 01:35:42,471 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Updated server config: isVoting: true, peers: [PeerInfo [id=leader573, address=pekko://test/user/leader573, votingState=VOTING], PeerInfo [id=follower574, address=pekko://test/user/follower574, votingState=VOTING], PeerInfo [id=follower576, address=pekko://test/user/follower576, votingState=NON_VOTING]] 01:35:42,471 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower575 in 101.3 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:42,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Recovery completed in 709.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:42,471 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=1, votedFor=null 01:35:42,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Local TermInfo store seeded with TermInfo{term=1} 01:35:42,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Snapshot capture initiated after recovery due to migrated messages 01:35:42,471 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:42,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:42,472 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: lastSequenceNumber prior to capture: 0 [WARN] [01/31/2025 13:35:42.472] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower576]: org.opendaylight.controller.cluster.common.actor.Monitor@5c7befa0 01:35:42,472 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: CaptureSnapshotReply received by actor 01:35:42,472 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader573, isVoting=true], ServerInfo[peerId=follower574, isVoting=true], ServerInfo[peerId=follower576, isVoting=false], ServerInfo[peerId=follower575, isVoting=true]]]] 01:35:42,472 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:42,472 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:42,473 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: Snapshot success - sequence number: -1 01:35:42,473 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: SnapshotComplete received 01:35:42,473 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower576 with journal batch size 1000 01:35:42,473 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: handled a snapshot response 01:35:42,473 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: handled a journal response 01:35:42,473 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: SnapshotOffer called. 01:35:42,473 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Moving last applied index from -1 to -1 01:35:42,473 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower576: Set currentTerm=1, votedFor=null 01:35:42,473 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Updated server config: isVoting: false, peers: [PeerInfo [id=leader573, address=pekko://test/user/leader573, votingState=VOTING], PeerInfo [id=follower575, address=pekko://test/user/follower575, votingState=VOTING], PeerInfo [id=follower574, address=pekko://test/user/follower574, votingState=VOTING]] 01:35:42,473 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower576 in 64.76 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Recovery completed in 677.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower576: Set currentTerm=1, votedFor=null 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Local TermInfo store seeded with TermInfo{term=1} 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Snapshot capture initiated after recovery due to migrated messages 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:42,474 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: lastSequenceNumber prior to capture: 0 [WARN] [01/31/2025 13:35:42.475] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader573]: org.opendaylight.controller.cluster.common.actor.Monitor@21b82038 01:35:42,475 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: CaptureSnapshotReply received by actor 01:35:42,475 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader573 with journal batch size 1000 01:35:42,475 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader573, isVoting=true], ServerInfo[peerId=follower575, isVoting=true], ServerInfo[peerId=follower574, isVoting=true], ServerInfo[peerId=follower576, isVoting=false]]]] 01:35:42,475 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: SnapshotOffer called. 01:35:42,475 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:42,475 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Moving last applied index from -1 to -1 01:35:42,475 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - leader573: Set currentTerm=1, votedFor=null 01:35:42,475 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:42,475 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: Snapshot success - sequence number: -1 01:35:42,475 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Updated server config: isVoting: true, peers: [PeerInfo [id=follower575, address=pekko://test/user/follower575, votingState=VOTING], PeerInfo [id=follower574, address=pekko://test/user/follower574, votingState=VOTING], PeerInfo [id=follower576, address=pekko://test/user/follower576, votingState=NON_VOTING]] 01:35:42,475 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader573 in 50.41 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:42,475 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: SnapshotComplete received 01:35:42,475 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: handled a snapshot response 01:35:42,476 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: handled a journal response 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Recovery completed in 665.1 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - leader573: Set currentTerm=1, votedFor=null 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Local TermInfo store seeded with TermInfo{term=1} 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Snapshot capture initiated after recovery due to migrated messages 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:42,476 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: lastSequenceNumber prior to capture: 0 01:35:42,477 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: CaptureSnapshotReply received by actor 01:35:42,477 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower575, isVoting=true], ServerInfo[peerId=follower574, isVoting=true], ServerInfo[peerId=follower576, isVoting=false], ServerInfo[peerId=leader573, isVoting=true]]]] 01:35:42,477 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:42,477 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:42,477 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Snapshot success - sequence number: -1 01:35:42,477 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: SnapshotComplete received 01:35:42,477 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: handled a journal response 01:35:42,477 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: handled a snapshot response 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.157:17355, Removed) 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Follower): Received ElectionTimeout - switching to Candidate 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate): Election: Candidate has following voting peers: [follower575, follower574] 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - leader573: Set currentTerm=2, votedFor=leader573 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate): Starting new election term 2 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate): Sending RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1} to peer follower575 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate): Sending RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1} to peer follower574 01:35:42,812 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574 (Follower): Cluster state: [] 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower574 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower574: Set currentTerm=2, votedFor=null 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574 (Follower): In requestVote: RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower574: Set currentTerm=2, votedFor=leader573 01:35:42,813 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Cluster state: [] 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=2, votedFor=null 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): In requestVote: RequestVote{term=2, candidateId=leader573, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower575: Set currentTerm=2, votedFor=leader573 01:35:42,813 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:42,813 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 01:35:42,813 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 01:35:42,813 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Election: Leader has following peers: [follower575, follower574, follower576] 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower575: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower574: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower576: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 01:35:42,814 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower574: Last sync leader does not match current leader leader573, need to catch up to -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower576 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader573, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 01:35:42,814 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower576: Set currentTerm=2, votedFor=null 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower574 updated: matchIndex: -1, nextIndex: 0 01:35:42,814 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower576: Last sync leader does not match current leader leader573, need to catch up to -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower574: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:42.814] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower574] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@31547d6e 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower576 updated: matchIndex: -1, nextIndex: 0 01:35:42,814 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower576: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower575: Last sync leader does not match current leader leader573, need to catch up to -1 01:35:42,814 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower575 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/31/2025 13:35:42.814] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower576] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@1677ec4d 01:35:42,815 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower575: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:42.815] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower575] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@1796bf05 01:35:42,829 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:42,829 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 01:35:42,829 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 01:35:42,829 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower575 starting 01:35:42,830 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower575: onShutDown 01:35:42,830 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower575 ending 01:35:42,830 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 01:35:42,830 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 01:35:42,830 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: onRequestLeadership RequestLeadership [requestedFollowerId=follower575, replyTo=Actor[pekko://test/user/$b#-198811915]] 01:35:42,830 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: Initiating leader transfer [WARN] [01/31/2025 13:35:42.830] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower575] received dead letter from TestActor[pekko://test/user/leader573]: LeaderTransitioning [leaderId=leader573] 01:35:42,830 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower574: onLeaderTransitioning: LeaderTransitioning [leaderId=leader573] 01:35:42,831 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower576: onLeaderTransitioning: LeaderTransitioning [leaderId=leader573] 01:35:42,831 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - leader573: pauseLeader successfully completed - doing transfer 01:35:42,831 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Attempting to transfer leadership 01:35:42,831 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower575: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:42.831] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower575] received dead letter from TestActor[pekko://test/user/leader573]: AppendEntries{term=2, leaderId=leader573, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:42,831 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower574: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,831 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower574: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:42.831] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower574] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@30a5b7b3 01:35:42,831 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Checking sendAppendEntries for follower follower576: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:42,831 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower576: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:42.831] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower576] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@5c90192d [WARN] [01/31/2025 13:35:42.932] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower575] received dead letter from TestActor[pekko://test/user/leader573]: AppendEntries{term=2, leaderId=leader573, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:42,933 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower574: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) 01:35:42,933 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower576: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:43.053] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower575] received dead letter from TestActor[pekko://test/user/leader573]: AppendEntries{term=2, leaderId=leader573, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:43,53 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower574: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) 01:35:43,53 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower576: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) 01:35:43,172 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - leader573: leader transfer aborted 01:35:43,172 PM [test-pekko.actor.default-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - leader573: Failed to transfer leadership in 342.1 ms 01:35:43,172 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573: LeadershipTransfer request from follower575 failed 01:35:43,173 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader573 (Leader): Leadership transfer expired [WARN] [01/31/2025 13:35:43.173] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower575] received dead letter from TestActor[pekko://test/user/leader573]: AppendEntries{term=2, leaderId=leader573, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:43,174 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower574: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) 01:35:43,174 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower576: Lagging 0 entries behind leader leader573 and reached -1 (of expected -1) 01:35:43,182 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Shutdown ending 01:35:43,182 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower574-notifier577#2009072482] 01:35:43,193 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower574-collector578#167615962] 01:35:43,193 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower574] 01:35:43,193 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower575-notifier579#1752188429] 01:35:43,193 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower575-collector580#-1999596040] 01:35:43,193 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower575] 01:35:43,194 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower576-notifier581#-1402005088] 01:35:43,194 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower576-collector582#-1159992979] 01:35:43,194 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower576] 01:35:43,194 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader573-notifier583#-518260253] 01:35:43,194 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader573-collector584#-868187054] 01:35:43,194 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader573] 01:35:43,195 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$b#-198811915] 01:35:43,195 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferOnShutDown starting 01:35:43,196 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/31/2025 13:35:43.198] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower586]: org.opendaylight.controller.cluster.common.actor.Monitor@2a3ffd12 01:35:43,198 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower586 with journal batch size 1000 01:35:43,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: SnapshotOffer called. 01:35:43,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Moving last applied index from -1 to -1 01:35:43,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower586: Set currentTerm=1, votedFor=null 01:35:43,198 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Updated server config: isVoting: true, peers: [PeerInfo [id=leader585, address=pekko://test/user/leader585, votingState=VOTING], PeerInfo [id=follower588, address=pekko://test/user/follower588, votingState=NON_VOTING], PeerInfo [id=follower587, address=pekko://test/user/follower587, votingState=VOTING]] 01:35:43,199 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower586 in 228.0 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/31/2025 13:35:43.199] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower587]: org.opendaylight.controller.cluster.common.actor.Monitor@78de2163 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Recovery completed in 782.9 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower586: Set currentTerm=1, votedFor=null 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Local TermInfo store seeded with TermInfo{term=1} 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Snapshot capture initiated after recovery due to migrated messages 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:43,199 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: lastSequenceNumber prior to capture: 0 01:35:43,200 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower587 with journal batch size 1000 01:35:43,200 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: SnapshotOffer called. 01:35:43,200 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Moving last applied index from -1 to -1 01:35:43,200 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: CaptureSnapshotReply received by actor 01:35:43,200 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=1, votedFor=null 01:35:43,200 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Updated server config: isVoting: true, peers: [PeerInfo [id=leader585, address=pekko://test/user/leader585, votingState=VOTING], PeerInfo [id=follower586, address=pekko://test/user/follower586, votingState=VOTING], PeerInfo [id=follower588, address=pekko://test/user/follower588, votingState=NON_VOTING]] 01:35:43,200 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower587 in 73.71 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:43,200 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader585, isVoting=true], ServerInfo[peerId=follower588, isVoting=false], ServerInfo[peerId=follower587, isVoting=true], ServerInfo[peerId=follower586, isVoting=true]]]] 01:35:43,200 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:43,200 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:43,200 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Snapshot success - sequence number: -1 01:35:43,201 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: SnapshotComplete received 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Recovery completed in 563.9 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=1, votedFor=null 01:35:43,201 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: handled a snapshot response 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Local TermInfo store seeded with TermInfo{term=1} 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Snapshot capture initiated after recovery due to migrated messages 01:35:43,201 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: handled a journal response 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:43,201 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: lastSequenceNumber prior to capture: 0 01:35:43,201 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: CaptureSnapshotReply received by actor 01:35:43,202 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader585, isVoting=true], ServerInfo[peerId=follower586, isVoting=true], ServerInfo[peerId=follower588, isVoting=false], ServerInfo[peerId=follower587, isVoting=true]]]] 01:35:43,202 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:43,202 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:43,202 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Snapshot success - sequence number: -1 [WARN] [01/31/2025 13:35:43.202] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower588]: org.opendaylight.controller.cluster.common.actor.Monitor@2d5d3f49 01:35:43,202 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: SnapshotComplete received 01:35:43,202 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: handled a snapshot response 01:35:43,202 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: handled a journal response 01:35:43,202 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower588 with journal batch size 1000 01:35:43,203 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: SnapshotOffer called. 01:35:43,203 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Moving last applied index from -1 to -1 01:35:43,203 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=1, votedFor=null 01:35:43,203 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Updated server config: isVoting: false, peers: [PeerInfo [id=leader585, address=pekko://test/user/leader585, votingState=VOTING], PeerInfo [id=follower586, address=pekko://test/user/follower586, votingState=VOTING], PeerInfo [id=follower587, address=pekko://test/user/follower587, votingState=VOTING]] 01:35:43,203 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower588 in 95.52 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Recovery completed in 679.7 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=1, votedFor=null 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Local TermInfo store seeded with TermInfo{term=1} 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Snapshot capture initiated after recovery due to migrated messages 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:43,203 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: lastSequenceNumber prior to capture: 0 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: CaptureSnapshotReply received by actor 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=leader585, isVoting=true], ServerInfo[peerId=follower586, isVoting=true], ServerInfo[peerId=follower587, isVoting=true], ServerInfo[peerId=follower588, isVoting=false]]]] 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Snapshot success - sequence number: -1 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: SnapshotComplete received 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: handled a snapshot response 01:35:43,204 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: handled a journal response [WARN] [01/31/2025 13:35:43.304] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader585]: org.opendaylight.controller.cluster.common.actor.Monitor@3a386fc7 01:35:43,305 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader585 with journal batch size 1000 01:35:43,305 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: SnapshotOffer called. 01:35:43,305 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Moving last applied index from -1 to -1 01:35:43,305 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader585: Set currentTerm=1, votedFor=null 01:35:43,305 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Updated server config: isVoting: true, peers: [PeerInfo [id=follower586, address=pekko://test/user/follower586, votingState=VOTING], PeerInfo [id=follower588, address=pekko://test/user/follower588, votingState=NON_VOTING], PeerInfo [id=follower587, address=pekko://test/user/follower587, votingState=VOTING]] 01:35:43,305 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader585 in 107.3 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Recovery completed in 801.9 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - leader585: Set currentTerm=1, votedFor=null 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Local TermInfo store seeded with TermInfo{term=1} 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Snapshot capture initiated after recovery due to migrated messages 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:43,306 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: lastSequenceNumber prior to capture: 0 01:35:43,308 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: CaptureSnapshotReply received by actor 01:35:43,308 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower586, isVoting=true], ServerInfo[peerId=follower588, isVoting=false], ServerInfo[peerId=follower587, isVoting=true], ServerInfo[peerId=leader585, isVoting=true]]]] 01:35:43,308 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:43,308 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:43,308 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Snapshot success - sequence number: -1 01:35:43,308 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: SnapshotComplete received 01:35:43,308 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: handled a snapshot response 01:35:43,308 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: handled a journal response 01:35:43,711 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.157:17355, Removed) 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Follower): Received ElectionTimeout - switching to Candidate 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate): Election: Candidate has following voting peers: [follower586, follower587] 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader585: Set currentTerm=2, votedFor=leader585 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate): Starting new election term 2 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate): Sending RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1} to peer follower586 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate): Sending RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1} to peer follower587 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:35:43,712 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:43,712 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Cluster state: [] 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:43,712 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Cluster state: [] 01:35:43,712 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:43,712 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower586: Set currentTerm=2, votedFor=null 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:43,712 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=2, votedFor=null 01:35:43,713 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): In requestVote: RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): In requestVote: RequestVote{term=2, candidateId=leader585, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:43,713 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower586: Set currentTerm=2, votedFor=leader585 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=2, votedFor=leader585 01:35:43,713 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Election: Leader has following peers: [follower586, follower588, follower587] 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower586: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower588: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower587: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:43,713 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 01:35:43,713 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 01:35:43,713 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=2, votedFor=null 01:35:43,714 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower587: Last sync leader does not match current leader leader585, need to catch up to -1 01:35:43,714 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower588: Last sync leader does not match current leader leader585, need to catch up to -1 [WARN] [01/31/2025 13:35:43.714] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower587] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@7be4c354 01:35:43,714 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower586: Last sync leader does not match current leader leader585, need to catch up to -1 [WARN] [01/31/2025 13:35:43.714] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower586] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@7414660c 01:35:43,714 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower587 updated: matchIndex: -1, nextIndex: 0 [WARN] [01/31/2025 13:35:43.714] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@51e878d6 01:35:43,714 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:43,714 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower588 updated: matchIndex: -1, nextIndex: 0 01:35:43,715 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:43,715 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower586 updated: matchIndex: -1, nextIndex: 0 01:35:43,715 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower586: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:43,761 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:43,761 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging starting 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Persist data SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower586: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower586 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Sending AppendEntries to follower follower586: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower588 01:35:43,761 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Sending AppendEntries to follower follower588: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,761 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,761 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Number of entries to be appended = 1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Number of entries to be appended = 1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Append entry to log zero 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Append entry to log zero 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Log size is now 1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower588, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 01:35:43,762 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader leader585 and reached -1 (of expected -1) 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Log size is now 1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower586, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 01:35:43,762 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower586: Lagging 0 entries behind leader leader585 and reached -1 (of expected -1) 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower587 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Sending AppendEntries to follower follower587: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower588 updated: matchIndex: 0, nextIndex: 1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower586 updated: matchIndex: 0, nextIndex: 1 01:35:43,762 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower586: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,767 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] [WARN] [01/31/2025 13:35:43.767] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@71cf6681 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 [WARN] [01/31/2025 13:35:43.767] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower586] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@471579e7 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Setting last applied to 0 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Moving last applied index from -1 to 0 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Applying state for log index 0 data zero 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: applyState called: zero 01:35:43,767 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Persisting ApplyJournalEntries with index=0 01:35:43,832 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower587 01:35:43,832 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Sending AppendEntries to follower follower587: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,952 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower587 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Sending AppendEntries to follower follower587: AppendEntries{term=2, leaderId=leader585, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Commit index set to 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Commit index set to 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader leader585 and reached 0 (of expected -1) 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] SyncStatusTracker - follower586: Lagging 0 entries behind leader leader585 and reached 0 (of expected -1) 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Setting last applied to 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Moving last applied index from -1 to 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Setting last applied to 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Applying state for log index 0 data zero 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Moving last applied index from -1 to 0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Applying state for log index 0 data zero 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: applyState called: zero 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: applyState called: zero 01:35:43,953 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: Persisting ApplyJournalEntries with index=0 01:35:43,953 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: Persisting ApplyJournalEntries with index=0 01:35:43,962 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging ending 01:35:43,962 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDownToLeaderAndVerifyLeadershipTransferToFollower1 starting 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: onShutDown 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: Initiating leader transfer 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorLeadershipTransferCohort - leader585: pauseLeader successfully completed - doing transfer 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Attempting to transfer leadership 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower586: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,962 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower587 01:35:43,963 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader leader585 and reached 0 (of expected -1) 01:35:43,963 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower588: onLeaderTransitioning: LeaderTransitioning [leaderId=leader585] 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): tryToCompleteLeadershipTransfer: followerId: follower586, matchIndex: 0, lastIndex: 0, isVoting: true 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Follower's log matches - sending ElectionTimeout 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower586: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): sendAppendEntries: 0 is present for follower follower587 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Leader transfer complete 01:35:43,963 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - leader585: leader transfer complete - waiting for new leader 01:35:43,963 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader leader585 and reached 0 (of expected -1) 01:35:43,963 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower586: Lagging 0 entries behind leader leader585 and reached 0 (of expected -1) 01:35:43,963 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower586: Lagging 0 entries behind leader leader585 and reached 0 (of expected -1) 01:35:43,963 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:43,963 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower): Received TimeoutNow - switching to Candidate 01:35:43,964 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate): Election: Candidate has following voting peers: [leader585, follower587] 01:35:43,964 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower586: Set currentTerm=3, votedFor=follower586 01:35:43,964 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate): Starting new election term 3 01:35:43,964 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate): Sending RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2} to peer leader585 01:35:43,964 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate): Sending RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2} to peer follower587 01:35:43,964 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:43,964 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Follower) :- Switching from behavior Follower to Candidate, election term: 3 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Cluster state: [] 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2} with higher term appears reachable, updating term. 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Term 3 in "RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2}" message is greater than follower's term 2 - updating term 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=3, votedFor=null 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): In requestVote: RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower587: Set currentTerm=3, votedFor=follower586 01:35:43,964 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 01:35:43,964 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Cluster state: [] 01:35:43,964 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2} with higher term appears reachable, updating term. 01:35:43,964 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Term 3 in "RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2}" message is greater than leader's term 2 - switching to Follower 01:35:43,964 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader585: Set currentTerm=3, votedFor=null 01:35:43,965 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): Leadership transfer in progress - processing RequestVote 01:35:43,965 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): In requestVote: RequestVote{term=3, candidateId=follower586, lastLogIndex=0, lastLogTerm=2} - currentTerm: 3, votedFor: null, lastIndex: 0, lastTerm: 2 01:35:43,965 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader585: Set currentTerm=3, votedFor=follower586 01:35:43,965 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586: onLeaderTransitioning: LeaderTransitioning [leaderId=leader585] 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Election: Leader has following peers: [leader585, follower588, follower587] 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Checking sendAppendEntries for follower leader585: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Checking sendAppendEntries for follower follower588: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Checking sendAppendEntries for follower follower587: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,965 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:35:43,965 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower588 (Follower): Term 3 in "AppendEntries{term=3, leaderId=follower586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 01:35:43,965 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower588: Set currentTerm=3, votedFor=null 01:35:43,966 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower588: Last sync leader does not match current leader follower586, need to catch up to -1 01:35:43,966 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower588 updated: matchIndex: 0, nextIndex: 1 01:35:43,966 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Checking sendAppendEntries for follower follower588: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,966 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader585 (Leader) :- Switching from behavior Leader to Follower, election term: 3 01:35:43,966 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - leader585: Last sync leader does not match current leader follower586, need to catch up to -1 [WARN] [01/31/2025 13:35:43.966] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@255acfe7 01:35:43,966 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): handleAppendEntriesReply - FollowerLogInformation for leader585 updated: matchIndex: 0, nextIndex: 1 01:35:43,966 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorLeadershipTransferCohort - leader585: leader changed to follower586 01:35:43,966 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Checking sendAppendEntries for follower leader585: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:43,967 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader follower586 and reached 0 (of expected -1) 01:35:43,966 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorLeadershipTransferCohort - leader585: Successfully transferred leadership to follower586 in 4.301 ms 01:35:43,967 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader585: leader transfer succeeded - sending PoisonPill [WARN] [01/31/2025 13:35:43.967] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower588] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@46bff50c 01:35:43,967 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - leader585: Lagging 0 entries behind leader follower586 and reached 0 (of expected -1) [WARN] [01/31/2025 13:35:43.967] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader585] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@4fee7680 [WARN] [01/31/2025 13:35:43.967] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/leader585] received dead letter: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@5d528963 01:35:44,483 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower588: Lagging 0 entries behind leader follower586 and reached 0 (of expected -1) 01:35:44,483 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower587: Last sync leader does not match current leader follower586, need to catch up to -1 01:35:44,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): sendAppendEntries: 0 is present for follower follower587 01:35:44,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Sending AppendEntries to follower follower587: AppendEntries{term=3, leaderId=follower586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,483 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=follower586, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Number of entries to be appended = 1 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Append entry to log zero 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Log size is now 1 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Commit index set to 0 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower587, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower587: Lagging 0 entries behind leader follower586 and reached 0 (of expected -1) 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587 (Follower): Setting last applied to 0 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Moving last applied index from -1 to 0 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Applying state for log index 0 data zero 01:35:44,484 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower587 updated: matchIndex: 0, nextIndex: 1 01:35:44,484 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower586 (Leader): Checking sendAppendEntries for follower follower587: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:44.483] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/leader585] received dead letter from TestActor[pekko://test/user/follower586]: AppendEntries{term=3, leaderId=follower586, prevLogIndex=0, prevLogTerm=2, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:44,484 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: applyState called: zero [WARN] [01/31/2025 13:35:44.484] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower587] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@1fa82cbb 01:35:44,485 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,485 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: Persisting ApplyJournalEntries with index=0 [WARN] [01/31/2025 13:35:44.485] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/follower587] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@43adea05 01:35:44,517 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: onLeaderTransitioning: LeaderTransitioning [leaderId=leader585] 01:35:44,517 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDownToLeaderAndVerifyLeadershipTransferToFollower1 ending 01:35:44,517 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower587 starting 01:35:44,518 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower587: onShutDown 01:35:44,518 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/follower587 ending 01:35:44,518 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferOnShutDown ending 01:35:44,518 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower586-notifier589#-359157213] 01:35:44,529 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower586-collector590#-651136395] 01:35:44,529 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower586] 01:35:44,530 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower587-notifier591#1585883388] 01:35:44,530 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower587-collector592#-1902151730] 01:35:44,530 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower587] 01:35:44,530 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower588-notifier593#737365150] 01:35:44,530 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower588-collector594#-285929914] 01:35:44,530 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower588] 01:35:44,531 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader585-notifier595#357343258] 01:35:44,531 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader585-collector596#-1658538261] 01:35:44,531 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader585] 01:35:44,532 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferSkippedOnShutdownWithNoFollowers starting [WARN] [01/31/2025 13:35:44.533] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader597]: org.opendaylight.controller.cluster.common.actor.Monitor@29054f87 01:35:44,533 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader597 with journal batch size 1000 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader597: Set currentTerm=0, votedFor=null 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Local TermInfo store seeded with TermInfo{term=0} 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Snapshot capture initiated after recovery due to migrated messages 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:44,535 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: lastSequenceNumber prior to capture: 0 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: CaptureSnapshotReply received by actor 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=0, electionVotedFor=null, ServerConfigPayload=null] 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Follower): Received TimeoutNow - switching to Candidate 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Candidate): Election: Candidate has following voting peers: [] 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - leader597: Set currentTerm=1, votedFor=leader597 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Candidate): Starting new election term 1 01:35:44,536 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Candidate): Received ElectionTimeout 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Leader): Election: Leader has following peers: [] 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: Snapshot success - sequence number: -1 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: SnapshotComplete received 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: handled a journal response 01:35:44,537 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: handled a snapshot response 01:35:44,586 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader597 starting 01:35:44,586 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader597: onShutDown 01:35:44,587 PM [main] [INFO] LeadershipTransferIntegrationTest - sendShutDown for pekko://test/user/leader597 ending 01:35:44,587 PM [main] [INFO] LeadershipTransferIntegrationTest - testLeaderTransferSkippedOnShutdownWithNoFollowers ending 01:35:44,587 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader597-collector601#1191324500] 01:35:44,598 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader597] 01:35:44,599 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Lagging starting 01:35:44,599 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/31/2025 13:35:44.600] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower603]: org.opendaylight.controller.cluster.common.actor.Monitor@718dadcd 01:35:44,601 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower603 with journal batch size 1000 01:35:44,602 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: SnapshotOffer called. 01:35:44,602 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Moving last applied index from -1 to -1 01:35:44,602 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower603: Set currentTerm=1, votedFor=null 01:35:44,602 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Updated server config: isVoting: true, peers: [PeerInfo [id=follower605, address=pekko://test/user/follower605, votingState=NON_VOTING], PeerInfo [id=follower604, address=pekko://test/user/follower604, votingState=VOTING], PeerInfo [id=leader602, address=pekko://test/user/leader602, votingState=VOTING]] 01:35:44,602 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower603 in 116.6 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Recovery completed in 1.115 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PersistenceTermInfoStore - follower603: Set currentTerm=1, votedFor=null 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Local TermInfo store seeded with TermInfo{term=1} 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Snapshot capture initiated after recovery due to migrated messages 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:44,603 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: lastSequenceNumber prior to capture: 0 [WARN] [01/31/2025 13:35:44.604] [test-pekko.actor.default-dispatcher-12] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower604]: org.opendaylight.controller.cluster.common.actor.Monitor@1c7ce09e 01:35:44,604 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower604 with journal batch size 1000 01:35:44,604 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: CaptureSnapshotReply received by actor 01:35:44,604 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower605, isVoting=false], ServerInfo[peerId=follower604, isVoting=true], ServerInfo[peerId=leader602, isVoting=true], ServerInfo[peerId=follower603, isVoting=true]]]] 01:35:44,605 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:44,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: SnapshotOffer called. 01:35:44,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Moving last applied index from -1 to -1 01:35:44,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=1, votedFor=null 01:35:44,605 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Updated server config: isVoting: true, peers: [PeerInfo [id=follower605, address=pekko://test/user/follower605, votingState=NON_VOTING], PeerInfo [id=leader602, address=pekko://test/user/leader602, votingState=VOTING], PeerInfo [id=follower603, address=pekko://test/user/follower603, votingState=VOTING]] 01:35:44,605 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower604 in 96.99 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:44,605 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Recovery completed in 814.6 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:44,606 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Snapshot success - sequence number: -1 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=1, votedFor=null 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Local TermInfo store seeded with TermInfo{term=1} 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Snapshot capture initiated after recovery due to migrated messages 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:44,606 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: lastSequenceNumber prior to capture: 0 01:35:44,606 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: SnapshotComplete received 01:35:44,606 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: handled a journal response 01:35:44,606 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: handled a snapshot response 01:35:44,606 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: CaptureSnapshotReply received by actor 01:35:44,606 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower605, isVoting=false], ServerInfo[peerId=leader602, isVoting=true], ServerInfo[peerId=follower603, isVoting=true], ServerInfo[peerId=follower604, isVoting=true]]]] 01:35:44,607 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:44,607 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:44,607 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: Snapshot success - sequence number: -1 01:35:44,607 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: SnapshotComplete received 01:35:44,607 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: handled a journal response 01:35:44,607 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: handled a snapshot response [WARN] [01/31/2025 13:35:44.611] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower605]: org.opendaylight.controller.cluster.common.actor.Monitor@e8db785 01:35:44,612 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower605 with journal batch size 1000 01:35:44,612 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: SnapshotOffer called. 01:35:44,613 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Moving last applied index from -1 to -1 01:35:44,613 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower605: Set currentTerm=1, votedFor=null 01:35:44,613 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Updated server config: isVoting: false, peers: [PeerInfo [id=follower604, address=pekko://test/user/follower604, votingState=VOTING], PeerInfo [id=leader602, address=pekko://test/user/leader602, votingState=VOTING], PeerInfo [id=follower603, address=pekko://test/user/follower603, votingState=VOTING]] 01:35:44,613 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower605 in 63.43 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/31/2025 13:35:44.613] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader602]: org.opendaylight.controller.cluster.common.actor.Monitor@542bd3dd 01:35:44,613 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader602 with journal batch size 1000 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: SnapshotOffer called. 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Moving last applied index from -1 to -1 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - leader602: Set currentTerm=1, votedFor=null 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Updated server config: isVoting: true, peers: [PeerInfo [id=follower605, address=pekko://test/user/follower605, votingState=NON_VOTING], PeerInfo [id=follower604, address=pekko://test/user/follower604, votingState=VOTING], PeerInfo [id=follower603, address=pekko://test/user/follower603, votingState=VOTING]] 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader602 in 52.49 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Recovery completed in 1.698 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower605: Set currentTerm=1, votedFor=null 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Local TermInfo store seeded with TermInfo{term=1} 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Snapshot capture initiated after recovery due to migrated messages 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:44,614 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: lastSequenceNumber prior to capture: 0 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Recovery completed in 1.164 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader602: Set currentTerm=1, votedFor=null 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Local TermInfo store seeded with TermInfo{term=1} 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Snapshot capture initiated after recovery due to migrated messages 01:35:44,615 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: CaptureSnapshotReply received by actor 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:44,615 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: lastSequenceNumber prior to capture: 0 01:35:44,615 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower604, isVoting=true], ServerInfo[peerId=leader602, isVoting=true], ServerInfo[peerId=follower603, isVoting=true], ServerInfo[peerId=follower605, isVoting=false]]]] 01:35:44,615 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Snapshot success - sequence number: -1 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: SnapshotComplete received 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: handled a journal response 01:35:44,616 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: CaptureSnapshotReply received by actor 01:35:44,616 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower605, isVoting=false], ServerInfo[peerId=follower604, isVoting=true], ServerInfo[peerId=follower603, isVoting=true], ServerInfo[peerId=leader602, isVoting=true]]]] 01:35:44,616 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: handled a snapshot response 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Snapshot success - sequence number: -1 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: SnapshotComplete received 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: handled a snapshot response 01:35:44,616 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: handled a journal response 01:35:44,952 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:44,952 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.157:17355, Removed) 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Follower): Received ElectionTimeout - switching to Candidate 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate): Election: Candidate has following voting peers: [follower604, follower603] 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - leader602: Set currentTerm=2, votedFor=leader602 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate): Starting new election term 2 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate): Sending RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1} to peer follower604 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate): Sending RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1} to peer follower603 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Cluster state: [] 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=2, votedFor=null 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Cluster state: [] 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower603: Set currentTerm=2, votedFor=null 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): In requestVote: RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PersistenceTermInfoStore - follower604: Set currentTerm=2, votedFor=leader602 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): In requestVote: RequestVote{term=2, candidateId=leader602, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:44,953 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PersistenceTermInfoStore - follower603: Set currentTerm=2, votedFor=leader602 01:35:44,953 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:44,954 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Election: Leader has following peers: [follower605, follower604, follower603] 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower605: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower604: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower603: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 01:35:44,955 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 01:35:44,955 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower604: Last sync leader does not match current leader leader602, need to catch up to -1 01:35:44,955 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] PersistenceTermInfoStore - follower605: Set currentTerm=2, votedFor=null 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower604 updated: matchIndex: -1, nextIndex: 0 01:35:44,955 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower605: Last sync leader does not match current leader leader602, need to catch up to -1 01:35:44,955 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:44.955] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower604] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@249822cf 01:35:44,956 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower603: Last sync leader does not match current leader leader602, need to catch up to -1 01:35:44,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower605 updated: matchIndex: -1, nextIndex: 0 01:35:44,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:44,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower603 updated: matchIndex: -1, nextIndex: 0 01:35:44,956 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower603: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:44.956] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower603] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@ec3baf4 [WARN] [01/31/2025 13:35:44.956] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower605] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@43909c67 01:35:44,968 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:44,968 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 01:35:44,969 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 01:35:44,969 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging starting 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Persist data SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): sendAppendEntries: 0 is present for follower follower605 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Sending AppendEntries to follower follower605: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): sendAppendEntries: 0 is present for follower follower604 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Sending AppendEntries to follower follower604: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower603: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): sendAppendEntries: 0 is present for follower follower603 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Sending AppendEntries to follower follower603: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Number of entries to be appended = 1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Append entry to log zero 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Number of entries to be appended = 1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Append entry to log zero 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Log size is now 1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower603, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower603: Lagging 0 entries behind leader leader602 and reached -1 (of expected -1) 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Log size is now 1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower605, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5} 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower603 updated: matchIndex: 0, nextIndex: 1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower605: Lagging 0 entries behind leader leader602 and reached -1 (of expected -1) 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower603: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower605 updated: matchIndex: 0, nextIndex: 1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:44,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,969 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:35:44,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Setting last applied to 0 01:35:44,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Moving last applied index from -1 to 0 01:35:44,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Applying state for log index 0 data zero [WARN] [01/31/2025 13:35:44.970] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower603] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@1e00faf7 01:35:44,970 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: persist complete SimpleReplicatedLogEntry [index=0, term=2, payload=zero] 01:35:44,970 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: applyState called: zero 01:35:44,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Persisting ApplyJournalEntries with index=0 [WARN] [01/31/2025 13:35:44.970] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower605] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6bbb4038 01:35:45,72 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): sendAppendEntries: 0 is present for follower follower604 01:35:45,72 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Sending AppendEntries to follower follower604: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Commit index set to 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower605: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605 (Follower): Setting last applied to 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Moving last applied index from -1 to 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Applying state for log index 0 data zero 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: applyState called: zero 01:35:45,73 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: Persisting ApplyJournalEntries with index=0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Commit index set to 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower603: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603 (Follower): Setting last applied to 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Moving last applied index from -1 to 0 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Applying state for log index 0 data zero 01:35:45,73 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: applyState called: zero 01:35:45,74 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: Persisting ApplyJournalEntries with index=0 01:35:45,120 PM [main] [INFO] LeadershipTransferIntegrationTest - sendPayloadWithFollower2Lagging ending 01:35:45,120 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 01:35:45,120 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 01:35:45,120 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: onRequestLeadership RequestLeadership [requestedFollowerId=follower604, replyTo=Actor[pekko://test/user/$c#-1934737379]] 01:35:45,120 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: Initiating leader transfer 01:35:45,121 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower605: onLeaderTransitioning: LeaderTransitioning [leaderId=leader602] 01:35:45,121 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - leader602: pauseLeader successfully completed - doing transfer 01:35:45,121 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Attempting to transfer leadership 01:35:45,121 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower603: onLeaderTransitioning: LeaderTransitioning [leaderId=leader602] 01:35:45,121 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower605: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:45,121 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower604: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:45,121 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): sendAppendEntries: 0 is present for follower follower604 01:35:45,121 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower604: onLeaderTransitioning: LeaderTransitioning [leaderId=leader602] 01:35:45,121 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Checking sendAppendEntries for follower follower603: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:35:45,121 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower605: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,121 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower603: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,192 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): sendAppendEntries: 0 is present for follower follower604 01:35:45,192 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Sending AppendEntries to follower follower604: AppendEntries{term=2, leaderId=leader602, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[SimpleReplicatedLogEntry [index=0, term=2, payload=zero]]} 01:35:45,312 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower605: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,312 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower603: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,432 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - leader602: leader transfer aborted 01:35:45,432 PM [test-pekko.actor.default-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - leader602: Failed to transfer leadership in 311.4 ms 01:35:45,432 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602: LeadershipTransfer request from follower604 failed 01:35:45,432 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader602 (Leader): Leadership transfer expired 01:35:45,432 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower603: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,432 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] SyncStatusTracker - follower605: Lagging 0 entries behind leader leader602 and reached 0 (of expected -1) 01:35:45,473 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithFollower2Lagging ending 01:35:45,473 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower603-notifier606#1640286588] 01:35:45,484 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower603-collector607#-2088352765] 01:35:45,484 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower603] 01:35:45,485 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower604-notifier608#-939852181] 01:35:45,485 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower604-collector609#251370620] 01:35:45,485 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower604] 01:35:45,485 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower605-notifier610#-831078707] 01:35:45,485 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower605-collector611#271991847] 01:35:45,486 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower605] 01:35:45,486 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader602-notifier612#507129912] 01:35:45,486 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader602-collector613#2103632860] 01:35:45,486 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader602] 01:35:45,486 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$c#-1934737379] 01:35:45,487 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithOtherFollowersDown starting 01:35:45,487 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting [WARN] [01/31/2025 13:35:45.489] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower615]: org.opendaylight.controller.cluster.common.actor.Monitor@5063ceaa 01:35:45,490 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower615 with journal batch size 1000 01:35:45,491 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: SnapshotOffer called. 01:35:45,491 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Moving last applied index from -1 to -1 01:35:45,491 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower615: Set currentTerm=1, votedFor=null 01:35:45,491 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Updated server config: isVoting: true, peers: [PeerInfo [id=follower616, address=pekko://test/user/follower616, votingState=VOTING], PeerInfo [id=follower617, address=pekko://test/user/follower617, votingState=NON_VOTING], PeerInfo [id=leader614, address=pekko://test/user/leader614, votingState=VOTING]] [WARN] [01/31/2025 13:35:45.492] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower616]: org.opendaylight.controller.cluster.common.actor.Monitor@22f9a02a 01:35:45,493 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower616 with journal batch size 1000 01:35:45,492 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower615 in 274.7 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:45,494 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: SnapshotOffer called. 01:35:45,494 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Moving last applied index from -1 to -1 01:35:45,494 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=1, votedFor=null 01:35:45,494 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Updated server config: isVoting: true, peers: [PeerInfo [id=follower615, address=pekko://test/user/follower615, votingState=VOTING], PeerInfo [id=follower617, address=pekko://test/user/follower617, votingState=NON_VOTING], PeerInfo [id=leader614, address=pekko://test/user/leader614, votingState=VOTING]] 01:35:45,494 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower616 in 132.6 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 [WARN] [01/31/2025 13:35:45.494] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower617]: org.opendaylight.controller.cluster.common.actor.Monitor@1c997be4 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Recovery completed in 3.105 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower615: Set currentTerm=1, votedFor=null 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Local TermInfo store seeded with TermInfo{term=1} 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Snapshot capture initiated after recovery due to migrated messages 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:45,495 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for follower617 with journal batch size 1000 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:45,495 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: lastSequenceNumber prior to capture: 0 01:35:45,495 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: SnapshotOffer called. 01:35:45,495 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Moving last applied index from -1 to -1 01:35:45,495 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=1, votedFor=null 01:35:45,495 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Updated server config: isVoting: false, peers: [PeerInfo [id=follower616, address=pekko://test/user/follower616, votingState=VOTING], PeerInfo [id=follower615, address=pekko://test/user/follower615, votingState=VOTING], PeerInfo [id=leader614, address=pekko://test/user/leader614, votingState=VOTING]] 01:35:45,495 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for follower617 in 37.76 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:45,495 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: CaptureSnapshotReply received by actor 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Recovery completed in 1.846 ms - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=1, votedFor=null 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Local TermInfo store seeded with TermInfo{term=1} 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Snapshot capture initiated after recovery due to migrated messages 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Recovery completed in 318.9 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=1, votedFor=null 01:35:45,496 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: lastSequenceNumber prior to capture: 0 01:35:45,496 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower616, isVoting=true], ServerInfo[peerId=follower617, isVoting=false], ServerInfo[peerId=leader614, isVoting=true], ServerInfo[peerId=follower615, isVoting=true]]]] 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Local TermInfo store seeded with TermInfo{term=1} 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Snapshot capture initiated after recovery due to migrated messages 01:35:45,496 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: lastSequenceNumber prior to capture: 0 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: CaptureSnapshotReply received by actor 01:35:45,496 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:45,496 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: Snapshot success - sequence number: -1 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower615, isVoting=true], ServerInfo[peerId=follower617, isVoting=false], ServerInfo[peerId=leader614, isVoting=true], ServerInfo[peerId=follower616, isVoting=true]]]] 01:35:45,496 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:45,496 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: SnapshotComplete received 01:35:45,496 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: handled a journal response 01:35:45,497 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: CaptureSnapshotReply received by actor 01:35:45,497 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower616, isVoting=true], ServerInfo[peerId=follower615, isVoting=true], ServerInfo[peerId=leader614, isVoting=true], ServerInfo[peerId=follower617, isVoting=false]]]] 01:35:45,497 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:45,497 PM [test-pekko.actor.default-dispatcher-12] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:45,497 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: Snapshot success - sequence number: -1 01:35:45,497 PM [test-pekko.actor.default-dispatcher-11] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:45,497 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: Snapshot success - sequence number: -1 01:35:45,497 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: SnapshotComplete received 01:35:45,497 PM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: SnapshotComplete received 01:35:45,497 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: handled a journal response 01:35:45,497 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615: handled a snapshot response 01:35:45,497 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: handled a journal response 01:35:45,497 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: handled a snapshot response 01:35:45,497 PM [test-pekko.actor.default-dispatcher-12] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower617: handled a snapshot response [WARN] [01/31/2025 13:35:45.497] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader614]: org.opendaylight.controller.cluster.common.actor.Monitor@3f3a8920 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Starting recovery for leader614 with journal batch size 1000 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: SnapshotOffer called. 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Moving last applied index from -1 to -1 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - leader614: Set currentTerm=1, votedFor=null 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Updated server config: isVoting: true, peers: [PeerInfo [id=follower616, address=pekko://test/user/follower616, votingState=VOTING], PeerInfo [id=follower615, address=pekko://test/user/follower615, votingState=VOTING], PeerInfo [id=follower617, address=pekko://test/user/follower617, votingState=NON_VOTING]] 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - Recovery snapshot applied for leader614 in 44.19 μs: snapshotIndex=-1, snapshotTerm=-1, journal-size=0 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Recovery completed in 598.2 μs - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - leader614: Set currentTerm=1, votedFor=null 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Local TermInfo store seeded with TermInfo{term=1} 01:35:45,498 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Snapshot capture initiated after recovery due to migrated messages 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: lastSequenceNumber prior to capture: 0 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: CaptureSnapshotReply received by actor 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=MockSnapshotState [state=[]], electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=follower616, isVoting=true], ServerInfo[peerId=follower615, isVoting=true], ServerInfo[peerId=follower617, isVoting=false], ServerInfo[peerId=leader614, isVoting=true]]]] 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: SaveSnapshotSuccess received for snapshot, sequenceNr: -1 01:35:45,499 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Snapshot success - sequence number: -1 01:35:45,500 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: SnapshotComplete received 01:35:45,500 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: handled a snapshot response 01:35:45,500 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: handled a journal response 01:35:45,832 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:45,832 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.157:17355, Removed) 01:35:45,832 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Follower): Received ElectionTimeout - switching to Candidate 01:35:45,832 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate): Election: Candidate has following voting peers: [follower616, follower615] 01:35:45,832 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - leader614: Set currentTerm=2, votedFor=leader614 01:35:45,833 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate): Starting new election term 2 01:35:45,833 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate): Sending RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1} to peer follower616 01:35:45,833 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate): Sending RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1} to peer follower615 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Cluster state: [] 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=2, votedFor=null 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615 (Follower): Cluster state: [] 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615 (Follower): Candidate in requestVote:RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower615 (Follower): Term 2 in "RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 1 - updating term 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower615: Set currentTerm=2, votedFor=null 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): In requestVote: RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=2, votedFor=leader614 01:35:45,833 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615 (Follower): In requestVote: RequestVote{term=2, candidateId=leader614, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 2, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower615: Set currentTerm=2, votedFor=leader614 01:35:45,833 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower615 (Follower): requestVote returning: RequestVoteReply{term=2, voteGranted=true} 01:35:45,833 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:35:45,834 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate): RaftRPC message received RequestVoteReply{term=2, voteGranted=true}, my term is 2 01:35:45,834 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate): handleRequestVoteReply: RequestVoteReply{term=2, voteGranted=true}, current voteCount: 1 01:35:45,834 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Election: Leader has following peers: [follower616, follower615, follower617] 01:35:45,834 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower616: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,835 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower616: Last sync leader does not match current leader leader614, need to catch up to -1 [WARN] [01/31/2025 13:35:45.835] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower616] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@679757d7 01:35:45,835 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower615: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,835 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower615: Last sync leader does not match current leader leader614, need to catch up to -1 [WARN] [01/31/2025 13:35:45.835] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower615] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@122a5f4 01:35:45,835 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower617: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,835 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower617 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader614, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 01:35:45,836 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - follower617: Set currentTerm=2, votedFor=null 01:35:45,836 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower617: Last sync leader does not match current leader leader614, need to catch up to -1 01:35:45,836 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 [WARN] [01/31/2025 13:35:45.836] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower617] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@6c13e6d7 01:35:45,836 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower616 updated: matchIndex: -1, nextIndex: 0 01:35:45,836 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower616: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,836 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower615 updated: matchIndex: -1, nextIndex: 0 01:35:45,836 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower615: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,837 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower617 updated: matchIndex: -1, nextIndex: 0 01:35:45,837 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower617: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,849 PM [main] [INFO] LeadershipTransferIntegrationTest - createRaftActors starting 01:35:45,849 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor starting 01:35:45,849 PM [main] [INFO] LeadershipTransferIntegrationTest - createRequestLeadershipResultCollectorActor ending 01:35:45,849 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower615] 01:35:45,860 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower617] 01:35:45,871 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader starting 01:35:45,871 PM [main] [INFO] LeadershipTransferIntegrationTest - sendFollower2RequestLeadershipTransferToLeader ending 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: onRequestLeadership RequestLeadership [requestedFollowerId=follower616, replyTo=Actor[pekko://test/user/$d#1473653891]] 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Initiating leader transfer 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorLeadershipTransferCohort - leader614: pauseLeader successfully completed - doing transfer 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Attempting to transfer leadership [WARN] [01/31/2025 13:35:45.871] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower615] received dead letter from TestActor[pekko://test/user/leader614]: LeaderTransitioning [leaderId=leader614] 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower616: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:45.871] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower617] received dead letter from TestActor[pekko://test/user/leader614]: LeaderTransitioning [leaderId=leader614] 01:35:45,871 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616: onLeaderTransitioning: LeaderTransitioning [leaderId=leader614] 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower615: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,871 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower617: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 [WARN] [01/31/2025 13:35:45.871] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower615] received dead letter from TestActor[pekko://test/user/leader614]: AppendEntries{term=2, leaderId=leader614, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} [WARN] [01/31/2025 13:35:45.872] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower617] received dead letter from TestActor[pekko://test/user/leader614]: AppendEntries{term=2, leaderId=leader614, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): tryToCompleteLeadershipTransfer: followerId: follower616, matchIndex: -1, lastIndex: -1, isVoting: true 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Follower's log matches - sending ElectionTimeout 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower616: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower615: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Checking sendAppendEntries for follower follower617: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Leader transfer complete 01:35:45,872 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - leader614: leader transfer complete - waiting for new leader [WARN] [01/31/2025 13:35:45.872] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower615] received dead letter from TestActor[pekko://test/user/leader614]: AppendEntries{term=2, leaderId=leader614, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:45,872 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower616: Lagging 0 entries behind leader leader614 and reached -1 (of expected -1) [WARN] [01/31/2025 13:35:45.872] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower617] received dead letter from TestActor[pekko://test/user/leader614]: AppendEntries{term=2, leaderId=leader614, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]} 01:35:45,872 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower616: Lagging 0 entries behind leader leader614 and reached -1 (of expected -1) 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower): Received TimeoutNow - switching to Candidate 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate): Election: Candidate has following voting peers: [follower615, leader614] 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PersistenceTermInfoStore - follower616: Set currentTerm=3, votedFor=follower616 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate): Starting new election term 3 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate): Sending RequestVote{term=3, candidateId=follower616, lastLogIndex=-1, lastLogTerm=-1} to peer follower615 [WARN] [01/31/2025 13:35:45.873] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower615] received dead letter from TestActor[pekko://test/user/follower616]: RequestVote{term=3, candidateId=follower616, lastLogIndex=-1, lastLogTerm=-1} 01:35:45,873 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate): Sending RequestVote{term=3, candidateId=follower616, lastLogIndex=-1, lastLogTerm=-1} to peer leader614 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Cluster state: [] 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Candidate in requestVote:RequestVote{term=3, candidateId=follower616, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term. 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Term 3 in "RequestVote{term=3, candidateId=follower616, lastLogIndex=-1, lastLogTerm=-1}" message is greater than leader's term 2 - switching to Follower 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - leader614: Set currentTerm=3, votedFor=null 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): Leadership transfer in progress - processing RequestVote 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): In requestVote: RequestVote{term=3, candidateId=follower616, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PersistenceTermInfoStore - leader614: Set currentTerm=3, votedFor=follower616 01:35:45,873 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader): requestVote returning: RequestVoteReply{term=3, voteGranted=true} 01:35:45,874 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - leader614 (Leader) :- Switching from behavior Leader to Follower, election term: 3 01:35:45,874 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Follower) :- Switching from behavior Follower to Candidate, election term: 3 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Leader): Election: Leader has following peers: [follower615, follower617, leader614] 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Leader): Checking sendAppendEntries for follower follower615: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Leader): Checking sendAppendEntries for follower follower617: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Leader): Checking sendAppendEntries for follower leader614: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - leader614: Last sync leader does not match current leader follower616, need to catch up to -1 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - leader614: leader changed to follower616 01:35:45,875 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Leader): handleAppendEntriesReply - FollowerLogInformation for leader614 updated: matchIndex: -1, nextIndex: 0 01:35:45,875 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - follower616 (Leader): Checking sendAppendEntries for follower leader614: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - leader614: Successfully transferred leadership to follower616 in 4.339 ms 01:35:45,875 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractRaftActorIntegrationTest$TestRaftActor - leader614: Leadership transferred successfully to follower616 [WARN] [01/31/2025 13:35:45.875] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower615] received dead letter from TestActor[pekko://test/user/follower616]: AppendEntries{term=3, leaderId=follower616, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} [WARN] [01/31/2025 13:35:45.875] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower617] received dead letter from TestActor[pekko://test/user/follower616]: AppendEntries{term=3, leaderId=follower616, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]} [WARN] [01/31/2025 13:35:45.875] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower616] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@dfcc14e [WARN] [01/31/2025 13:35:45.875] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/leader614] unhandled message from Actor[pekko://test/deadLetters]: org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus@54971c53 01:35:45,922 PM [main] [INFO] LeadershipTransferIntegrationTest - testRequestLeadershipTransferToFollower2WithOtherFollowersDown ending 01:35:45,922 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower615-notifier618#194117772] 01:35:45,933 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower615-collector619#454150424] 01:35:45,933 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower616-notifier620#-97811038] 01:35:45,933 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower616-collector621#-1633745911] 01:35:45,934 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower616] 01:35:45,934 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower617-notifier622#-330270463] 01:35:45,934 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower617-collector623#556601397] 01:35:45,934 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader614-notifier624#557594361] 01:35:45,934 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader614-collector625#-1191426] 01:35:45,934 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader614] 01:35:45,935 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/$d#1473653891] [INFO] [01/31/2025 13:35:45.936] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:35:45,936 PM [test-pekko.actor.default-dispatcher-6] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:35:45,937 PM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - Exiting completed 01:35:45,937 PM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - Shutting down... 01:35:45,938 PM [test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.157:17355] - Successfully shut down [WARN] [01/31/2025 13:35:45.938] [test-pekko.actor.default-dispatcher-15] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.170.157:17355) [INFO] [01/31/2025 13:35:45.937] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - Exiting completed [INFO] [01/31/2025 13:35:45.937] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - Shutting down... [INFO] [01/31/2025 13:35:45.937] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.157:17355] - Successfully shut down [INFO] [01/31/2025 13:35:45.939] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.170.157:17355/system/remoting-terminator] Shutting down remote daemon. 01:35:45,939 PM [test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. [INFO] [01/31/2025 13:35:45.939] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.170.157:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports. 01:35:45,939 PM [test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports.