[DEBUG] [06/27/2025 14:30:37.404] [main] [EventStream(pekko://test)] logger log1-TestEventListener started
02:30:37,412 PM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started
[DEBUG] [06/27/2025 14:30:37.413] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started
[DEBUG] [06/27/2025 14:30:37.413] [main] [EventStream(pekko://test)] Default Loggers started
02:30:37,444 PM [test-pekko.actor.default-dispatcher-7] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.8:17355] with UID [-6057921203309239354]
[INFO] [06/27/2025 14:30:37.443] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.170.8:17355] with UID [-6057921203309239354]
02:30:37,444 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Starting up, Pekko version [1.0.3] ...
[INFO] [06/27/2025 14:30:37.444] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Starting up, Pekko version [1.0.3] ...
[INFO] [06/27/2025 14:30:37.448] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Registered cluster JMX MBean [pekko:type=Cluster]
[INFO] [06/27/2025 14:30:37.448] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Started up successfully
02:30:37,448 PM [test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Registered cluster JMX MBean [pekko:type=Cluster]
02:30:37,448 PM [test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Started up successfully
[INFO] [06/27/2025 14:30:37.449] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing
[INFO] [06/27/2025 14:30:37.449] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining
02:30:37,449 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing
02:30:37,449 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining
02:30:37,463 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerLeaderStateChanges
[WARN] [06/27/2025 14:30:37.467] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower17]: org.opendaylight.controller.cluster.common.actor.Monitor@279e4b35
[INFO] [pekkoDeadLetter][06/27/2025 14:30:37.467] [test-pekko.actor.default-dispatcher-8] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower17] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:37,468 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower17] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:37,472 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - follower17: Starting recovery with journal batch size 1000
02:30:37,472 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower17: started executor
02:30:37,472 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower17: started executor
02:30:37,473 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower17: metadata journal open: replayFrom=1 applyTo=0
02:30:37,474 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - follower17: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:37,474 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - follower17: journal open: applyTo=0
02:30:37,474 PM [follower17-writer-7] [DEBUG] JournalWriteTask - follower17: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6aa34778 started
02:30:37,474 PM [follower17-writer-7] [DEBUG] JournalWriteTask - follower17: waiting for more work
02:30:37,476 PM [test-pekko.actor.default-dispatcher-7] [INFO] RaftActor - leader16: Starting recovery with journal batch size 1000
02:30:37,476 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader16: started executor
02:30:37,476 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - leader16: started executor
02:30:37,476 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader16] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [06/27/2025 14:30:37.477] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader16]: org.opendaylight.controller.cluster.common.actor.Monitor@6a248a96
[INFO] [pekkoDeadLetter][06/27/2025 14:30:37.476] [test-pekko.actor.default-dispatcher-8] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader16] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:37,481 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader16: metadata journal open: replayFrom=1 applyTo=0
02:30:37,482 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryJournalV1 - leader16: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:37,482 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRaftStorage - leader16: journal open: applyTo=0
02:30:37,482 PM [leader16-writer-8] [DEBUG] JournalWriteTask - leader16: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@71be4f85 started
02:30:37,483 PM [leader16-writer-8] [DEBUG] JournalWriteTask - leader16: waiting for more work
02:30:37,485 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest16505186010764248695/follower17/TermInfo.properties does not exist
02:30:37,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest16505186010764248695/leader16/TermInfo.properties does not exist
02:30:37,485 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - follower17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest16505186010764248695/follower17/journal-v1-1.log
02:30:37,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader16: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest16505186010764248695/leader16/journal-v1-1.log
02:30:37,485 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - follower17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest16505186010764248695/follower17/metadata-v1-1.log
02:30:37,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader16: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest16505186010764248695/leader16/metadata-v1-1.log
02:30:37,485 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - follower17: no eligible files found
02:30:37,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader16: no eligible files found
02:30:37,485 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - follower17: Set currentTerm=1, votedFor=leader16
02:30:37,485 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader16: Set currentTerm=1, votedFor=leader16
02:30:37,485 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - follower17: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:37,486 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PeerInfos - follower17: Updated server config: isVoting: false, peers: [PeerInfo [id=leader16, address=pekko://test/user/leader16, votingState=VOTING]]
02:30:37,486 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRecovery - leader16: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:37,486 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - follower17: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:37,486 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PeerInfos - leader16: Updated server config: isVoting: true, peers: [PeerInfo [id=follower17, address=pekko://test/user/follower17, votingState=NON_VOTING]]
02:30:37,486 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader16: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:37,520 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - follower17: Set currentTerm=1, votedFor=leader16
02:30:37,520 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - follower17: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader16}
02:30:37,520 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - follower17: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@113456a5
02:30:37,536 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader16: Set currentTerm=1, votedFor=leader16
02:30:37,536 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader16: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader16}
02:30:37,536 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader16: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@35b7dfd3
02:30:38,5 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:38,5 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader16 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:38,5 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader16 (Follower): Received ElectionTimeout - switching to Candidate
02:30:38,5 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader16 (Candidate): Election: Candidate has following voting peers: []
02:30:38,10 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader16: Set currentTerm=2, votedFor=leader16
02:30:38,10 PM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader16 (Candidate): Starting new election term 2
02:30:38,10 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader16 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
02:30:38,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader16 (Candidate): Received ElectionTimeout
02:30:38,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader16 (Leader): Election: Leader has following peers: [follower17]
02:30:38,11 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader16 (Leader): Checking sendAppendEntries for follower follower17: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:38,11 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader16 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
02:30:38,11 PM [test-pekko.actor.default-dispatcher-7] [INFO] Follower - follower17 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader16, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term
02:30:38,14 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower17: Set currentTerm=2, votedFor=null
02:30:38,14 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower17: Last sync leader does not match current leader leader16, need to catch up to -1
02:30:38,14 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader16 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower17 updated: matchIndex: -1, nextIndex: 0
02:30:38,14 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader16 (Leader): Checking sendAppendEntries for follower follower17: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:38,16 PM [test-pekko.actor.default-dispatcher-7] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower17] was unhandled. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [06/27/2025 14:30:38.016] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower17] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower17, initialSyncDone=false]
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.016] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower17] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower17] was unhandled. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,47 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:38,365 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower17: Lagging 0 entries behind leader leader16 and reached -1 (of expected -1)
02:30:38,366 PM [test-pekko.actor.default-dispatcher-8] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower17] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [06/27/2025 14:30:38.366] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/follower17] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower17, initialSyncDone=true]
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.366] [test-pekko.actor.default-dispatcher-8] [pekko://test/user/follower17] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower17] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,400 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/roleChangeNotifier19#1843518160]
02:30:38,412 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower17-collector20#-1725355822]
02:30:38,414 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower17]
02:30:38,414 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - follower17: terminating thread VirtualThread[#330,follower17-writer-7]/waiting
02:30:38,415 PM [follower17-writer-7] [DEBUG] JournalWriteTask - follower17: received 1 entries in 940.8 ms
02:30:38,415 PM [follower17-writer-7] [DEBUG] JournalWriteTask - follower17: persisted 1 entries in 63.65 μs
02:30:38,415 PM [follower17-writer-7] [DEBUG] JournalWriteTask - follower17: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6aa34778 stopped
02:30:38,415 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower17: journal closed
02:30:38,415 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower17: journal closed
02:30:38,415 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower17: stopped executor with 0 remaining tasks
02:30:38,415 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower17: stopped executor with 0 remaining tasks
02:30:38,416 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader16-collector21#2027793019]
02:30:38,416 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader16]
02:30:38,416 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader16: terminating thread VirtualThread[#333,leader16-writer-8]/waiting
02:30:38,417 PM [leader16-writer-8] [DEBUG] JournalWriteTask - leader16: received 1 entries in 934.1 ms
02:30:38,417 PM [leader16-writer-8] [DEBUG] JournalWriteTask - leader16: persisted 1 entries in 35.72 μs
02:30:38,417 PM [leader16-writer-8] [DEBUG] JournalWriteTask - leader16: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@71be4f85 stopped
02:30:38,417 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader16: journal closed
02:30:38,417 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader16: journal closed
02:30:38,417 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader16: stopped executor with 0 remaining tasks
02:30:38,417 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader16: stopped executor with 0 remaining tasks
02:30:38,419 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithLessLeaderLogEntriesAfterNonPersistentLeaderRestart starting
[WARN] [06/27/2025 14:30:38.424] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower23]: org.opendaylight.controller.cluster.common.actor.Monitor@4821d081
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.424] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower23] to Actor[pekko://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,424 PM [test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower23] to Actor[pekko://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,426 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - follower23: Starting recovery with journal batch size 1000
02:30:38,426 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower23: started executor
02:30:38,426 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower23: started executor
02:30:38,427 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower23: metadata journal open: replayFrom=1 applyTo=0
02:30:38,435 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - follower23: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:38,435 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - follower23: journal open: applyTo=0
02:30:38,435 PM [follower23-writer-9] [DEBUG] JournalWriteTask - follower23: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@14cbfba8 started
02:30:38,435 PM [follower23-writer-9] [DEBUG] JournalWriteTask - follower23: waiting for more work
02:30:38,437 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/TermInfo.properties does not exist
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/journal-v1-1.log
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/metadata-v1-1.log
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower23: no eligible files found
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower23: Set currentTerm=1, votedFor=leader22
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - follower23: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PeerInfos - follower23: Updated server config: isVoting: false, peers: [PeerInfo [id=leader22, address=pekko://test/user/leader22, votingState=VOTING]]
02:30:38,438 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower23: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
[WARN] [06/27/2025 14:30:38.439] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader22]: org.opendaylight.controller.cluster.common.actor.Monitor@2c264852
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.439] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader22] to Actor[pekko://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,439 PM [test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader22] to Actor[pekko://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,440 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader22: Starting recovery with journal batch size 1000
02:30:38,440 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: started executor
02:30:38,440 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: started executor
02:30:38,441 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader22: metadata journal open: replayFrom=1 applyTo=0
02:30:38,441 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader22: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:38,441 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader22: journal open: applyTo=0
02:30:38,441 PM [leader22-writer-10] [DEBUG] JournalWriteTask - leader22: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@51f96919 started
02:30:38,441 PM [leader22-writer-10] [DEBUG] JournalWriteTask - leader22: waiting for more work
02:30:38,443 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/TermInfo.properties does not exist
02:30:38,443 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/journal-v1-1.log
02:30:38,443 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/metadata-v1-1.log
02:30:38,443 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: no eligible files found
02:30:38,443 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=1, votedFor=leader22
02:30:38,444 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRecovery - leader22: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:38,444 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PeerInfos - leader22: Updated server config: isVoting: true, peers: [PeerInfo [id=follower23, address=pekko://test/user/follower23, votingState=NON_VOTING]]
02:30:38,444 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader22: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:38,446 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower23: Set currentTerm=1, votedFor=leader22
02:30:38,446 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower23: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader22}
02:30:38,446 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower23: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3f40a3f6
02:30:38,450 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=1, votedFor=leader22
02:30:38,450 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader22: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader22}
02:30:38,450 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader22: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@6823c01a
02:30:38,885 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:38,885 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - leader22 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:38,885 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - leader22 (Follower): Received ElectionTimeout - switching to Candidate
02:30:38,885 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Candidate - leader22 (Candidate): Election: Candidate has following voting peers: []
02:30:38,890 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=2, votedFor=leader22
02:30:38,890 PM [test-pekko.actor.default-dispatcher-8] [INFO] Candidate - leader22 (Candidate): Starting new election term 2
02:30:38,890 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActorBehavior - leader22 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
02:30:38,890 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Candidate - leader22 (Candidate): Received ElectionTimeout
02:30:38,890 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader22 (Leader): Election: Leader has following peers: [follower23]
02:30:38,890 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:38,891 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActorBehavior - leader22 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
02:30:38,891 PM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower23 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader22, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term
02:30:38,893 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower23: Set currentTerm=2, votedFor=null
02:30:38,893 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower23: Last sync leader does not match current leader leader22, need to catch up to -1
02:30:38,894 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader22 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower23 updated: matchIndex: -1, nextIndex: 0
02:30:38,894 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:38,894 PM [test-pekko.actor.default-dispatcher-6] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower23] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [06/27/2025 14:30:38.894] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower23] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower23, initialSyncDone=false]
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.894] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower23] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower23] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,906 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader22: Persist data index=0 term=2 command=zero
02:30:38,906 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader22: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=true} 
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - leader22 (Leader): Setting last applied to 0
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader22: Moving last applied index from -1 to 0
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader22: Applying state for log index 0 data zero
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - leader22: applyState called: zero
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): sendAppendEntries: 0 is present for follower follower23
02:30:38,907 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Sending AppendEntries to follower follower23: AppendEntries{term=2, leaderId=leader22, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=true}]}
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader22: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=true}
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader22: Persist data index=1 term=2 command=one
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader22, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}]}
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader22: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=true} 
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Number of entries to be appended = 1
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Append entry to log zero
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - leader22 (Leader): Setting last applied to 1
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower23: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=false} 
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader22: Moving last applied index from 0 to 1
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader22: Applying state for log index 1 data one
02:30:38,908 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - leader22: applyState called: one
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower23: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=false}
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Log size is now 1
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Commit index set to 0
02:30:38,908 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower23, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower23: Lagging 0 entries behind leader leader22 and reached 0 (of expected -1)
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): sendAppendEntries: 0 is present for follower follower23
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower23 (Follower): Setting last applied to 0
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower23: Moving last applied index from -1 to 0
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower23: Applying state for log index 0 data zero
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower23: applyState called: zero
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Sending AppendEntries to follower follower23: AppendEntries{term=2, leaderId=leader22, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=true}]}
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader22: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=true}
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower23 updated: matchIndex: 0, nextIndex: 1
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader22, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - leader22: fakeSnapshot purging log to 0 for term 2
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Number of entries to be appended = 2
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): matchEntry JournaledLogEntry{index=0, term=2, command=zero, pending=false} is present: existingEntryTerm: 2
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): After cleanup, lastIndex: 0, entries to be added from: 1
02:30:38,909 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0
02:30:38,909 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Append entry to log one
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): sendAppendEntries: 1 is present for follower follower23
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower23: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=false} 
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower23: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=false}
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Sending AppendEntries to follower follower23: AppendEntries{term=2, leaderId=leader22, prevLogIndex=0, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Log size is now 2
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Commit index set to 1
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower23, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower23: Lagging 0 entries behind leader leader22 and reached 1 (of expected -1)
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader22: Persist data index=2 term=2 command=two
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower23 (Follower): Setting last applied to 1
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower23: Moving last applied index from 0 to 1
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower23: Applying state for log index 1 data one
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader22: Append log entry and persist JournaledLogEntry{index=2, term=2, command=two, pending=true} 
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower23: applyState called: one
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - leader22 (Leader): Setting last applied to 2
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractReplicatedLog - leader22: Moving last applied index from 1 to 2
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader22: Applying state for log index 2 data two
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - leader22: applyState called: two
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: 0
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): sendAppendEntries: 1 is present for follower follower23
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader22, prevLogIndex=0, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Number of entries to be appended = 1
02:30:38,910 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): matchEntry JournaledLogEntry{index=1, term=2, command=one, pending=false} is present: existingEntryTerm: 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): After cleanup, lastIndex: 1, entries to be added from: 1
02:30:38,910 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Sending AppendEntries to follower follower23: AppendEntries{term=2, leaderId=leader22, prevLogIndex=0, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}, JournaledLogEntry{index=2, term=2, command=two, pending=true}]}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): Log size is now 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower23, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower23: Lagging 0 entries behind leader leader22 and reached 1 (of expected -1)
02:30:38,911 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower23: fakeSnapshot purging log to 0 for term 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - leader22: persist complete JournaledLogEntry{index=2, term=2, command=two, pending=true}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower23 updated: matchIndex: 1, nextIndex: 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - leader22: fakeSnapshot purging log to 1 for term 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader22, prevLogIndex=0, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}, JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): Number of entries to be appended = 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): matchEntry JournaledLogEntry{index=1, term=2, command=one, pending=false} is present: existingEntryTerm: 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): After cleanup, lastIndex: 1, entries to be added from: 1
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): sendAppendEntries: 2 is present for follower follower23
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): Append entry to log two
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower23: Append log entry and persist JournaledLogEntry{index=2, term=2, command=two, pending=false} 
02:30:38,911 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Sending AppendEntries to follower follower23: AppendEntries{term=2, leaderId=leader22, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower23: persist complete JournaledLogEntry{index=2, term=2, command=two, pending=false}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): Log size is now 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): Commit index set to 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower23, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower23: Lagging 0 entries behind leader leader22 and reached 2 (of expected -1)
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower23 (Follower): Setting last applied to 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower23: Moving last applied index from 1 to 2
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower23: Applying state for log index 2 data two
02:30:38,911 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower23: applyState called: two
02:30:38,912 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): sendAppendEntries: 2 is present for follower follower23
02:30:38,912 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower23 updated: matchIndex: 2, nextIndex: 3
02:30:38,912 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader22, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): Number of entries to be appended = 1
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): matchEntry JournaledLogEntry{index=2, term=2, command=two, pending=false} is present: existingEntryTerm: 2
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): After cleanup, lastIndex: 2, entries to be added from: 1
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): Log size is now 2
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower23 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower23, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower23: Lagging 0 entries behind leader leader22 and reached 2 (of expected -1)
02:30:38,912 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower23: fakeSnapshot purging log to 1 for term 2
02:30:38,913 PM [test-pekko.actor.default-dispatcher-15] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower23] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[WARN] [06/27/2025 14:30:38.912] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower23] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower23, initialSyncDone=true]
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.912] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower23] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower23] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,969 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRaftStorage - leader22: terminating thread VirtualThread[#335,leader22-writer-10]/waiting
02:30:38,969 PM [leader22-writer-10] [DEBUG] JournalWriteTask - leader22: received 1 entries in 527.6 ms
02:30:38,969 PM [leader22-writer-10] [DEBUG] JournalWriteTask - leader22: persisted 1 entries in 45.23 μs
02:30:38,969 PM [leader22-writer-10] [DEBUG] JournalWriteTask - leader22: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@51f96919 stopped
02:30:38,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader22: journal closed
02:30:38,970 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader22: journal closed
02:30:38,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: stopped executor with 0 remaining tasks
02:30:38,970 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: stopped executor with 0 remaining tasks
[WARN] [06/27/2025 14:30:38.973] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader22]: org.opendaylight.controller.cluster.common.actor.Monitor@43f2e45
[INFO] [pekkoDeadLetter][06/27/2025 14:30:38.974] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader22] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,974 PM [test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader22] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:38,975 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader22: Starting recovery with journal batch size 1000
02:30:38,975 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: started executor
02:30:38,975 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader22: started executor
02:30:38,976 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader22: metadata journal open: replayFrom=1 applyTo=0
02:30:38,977 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryJournalV1 - leader22: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:38,977 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRaftStorage - leader22: journal open: applyTo=0
02:30:38,977 PM [leader22-writer-11] [DEBUG] JournalWriteTask - leader22: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@21917cd9 started
02:30:38,977 PM [leader22-writer-11] [DEBUG] JournalWriteTask - leader22: waiting for more work
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=2, votedFor=leader22
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/TermInfo.properties
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/journal-v1-1.log
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/metadata-v1-1.log
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader22: no eligible files found
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=1, votedFor=leader22
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - leader22: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PeerInfos - leader22: Updated server config: isVoting: true, peers: [PeerInfo [id=follower23, address=pekko://test/user/follower23, votingState=NON_VOTING]]
02:30:38,979 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - leader22: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:38,980 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - leader22: restoring local TermInfo{term=2, votedFor=leader22}
02:30:38,980 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=2, votedFor=leader22
02:30:38,980 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader22: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3d479477
02:30:39,474 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader22 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:39,475 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader22 (Follower): Received ElectionTimeout - switching to Candidate
02:30:39,475 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader22 (Candidate): Election: Candidate has following voting peers: []
02:30:39,479 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader22: Set currentTerm=3, votedFor=leader22
02:30:39,479 PM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader22 (Candidate): Starting new election term 3
02:30:39,479 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader22 (Follower) :- Switching from behavior Follower to Candidate, election term: 3
02:30:39,480 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader22 (Candidate): Received ElectionTimeout
02:30:39,480 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): Election: Leader has following peers: [follower23]
02:30:39,480 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:39,480 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader22 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader22: Persist data index=0 term=3 command=zero-1
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader22: Append log entry and persist JournaledLogEntry{index=0, term=3, command=zero-1, pending=true} 
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): Replicate message: identifier: MockIdentifier{value=zero-1}, logIndex: 0, isSendImmediate: true
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader22 (Leader): Setting last applied to 0
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader22: Moving last applied index from -1 to 0
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader22: Applying state for log index 0 data zero-1
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader22: applyState called: zero-1
02:30:39,489 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader22: persist complete JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader22: Persist data index=1 term=3 command=one-1
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader22: Append log entry and persist JournaledLogEntry{index=1, term=3, command=one-1, pending=true} 
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): Replicate message: identifier: MockIdentifier{value=one-1}, logIndex: 1, isSendImmediate: true
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader22 (Leader): Setting last applied to 1
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader22: Moving last applied index from 0 to 1
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader22: Applying state for log index 1 data one-1
02:30:39,490 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader22: applyState called: one-1
02:30:39,491 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): Checking sendAppendEntries for follower follower23: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:39,491 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:39,491 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader22: persist complete JournaledLogEntry{index=1, term=3, command=one-1, pending=true}
02:30:39,595 PM [test-pekko.actor.default-dispatcher-8] [INFO] Follower - follower23 (Follower): Term 3 in "AppendEntries{term=3, leaderId=leader22, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term
02:30:39,600 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - follower23: Set currentTerm=3, votedFor=null
02:30:39,600 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower23: Lagging -3 entries behind leader leader22 and reached 2 (of expected -1)
02:30:39,601 PM [test-pekko.actor.default-dispatcher-6] [INFO] AbstractLeader - leader22 (Leader): handleAppendEntriesReply: follower follower23 lastIndex 2 is ahead of our lastIndex 1 (snapshotIndex -1, snapshotTerm -1) - forcing install snaphot
02:30:39,601 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader22: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=3, lastIndex=1, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower23
02:30:39,601 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader22: lastSequenceNumber prior to capture: -1
02:30:39,601 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader22: support() called
02:30:39,610 PM [leader22-%d617949] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:39,610 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:39,610 PM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader22: Persising snapshot at EntryInfo[index=1, term=3]/EntryInfo[index=1, term=3]
02:30:39,612 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader22: support() called
02:30:39,613 PM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader22: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1
02:30:39,614 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/snapshot-00000000685EAB0F-24953659.tmp
02:30:39,614 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): sendInstallSnapshot
02:30:39,615 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] LeaderInstallSnapshotState - leader22 (Leader): Snapshot 298 bytes, total chunks to send: 1
02:30:39,615 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] LeaderInstallSnapshotState - leader22 (Leader): Next chunk: total length=298, offset=0, size=298, hashCode=-1896618261
02:30:39,615 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): next snapshot chunk size for follower follower23: 298
02:30:39,615 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleInstallSnapshot: InstallSnapshot{term=3, leaderId=leader22, lastIncludedIndex=1, lastIncludedTerm=3, datasize=298, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=follower23, isVoting=false], ServerInfo[peerId=leader22, isVoting=true]]}}
02:30:39,616 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower23: Lagging -1 entries behind leader leader22 and reached 2 (of expected -1)
02:30:39,616 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotTracker - follower23 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
02:30:39,616 PM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower23 (Follower): Snapshot received from leader: leader22
02:30:39,617 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - follower23: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=298}}
02:30:39,617 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower23: support() called
02:30:39,617 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): InstallSnapshot sent to follower Vector(user, follower23), Chunk: 1/1
02:30:39,619 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower23: Converted InstallSnapshot from leader: leader22 to state
02:30:39,619 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower23: lastSequenceNumber prior to persisting applied snapshot: -1
02:30:39,619 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower23: support() called
02:30:39,620 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/snapshot-00000000685EAB0F-24F4DAEE.tmp
02:30:39,621 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/snapshot-00000000685EAB0F-24953659.v1
02:30:39,621 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/snapshot-00000000685EAB0F-24953659.v1
02:30:39,622 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/TermInfo.properties
02:30:39,622 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/journal-v1-1.log
02:30:39,622 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/metadata-v1-1.log
02:30:39,622 PM [leader22-%d617950] [DEBUG] RaftStorage - leader22: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:39.613758553Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/leader22/snapshot-00000000685EAB0F-24953659.v1, size=88, entryCompress=NONE, stateCompress=NONE}
02:30:39,622 PM [leader22-%d617950] [DEBUG] EntryStoreCompleter - leader22: 1 completion(s) pending
02:30:39,622 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryStoreCompleter - leader22: running 1 completion(s)
02:30:39,622 PM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader22: snapshot is durable as of 2025-06-27T14:30:39.613758553Z
02:30:39,622 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - leader22: Snapshot success -  sequence number: -1
02:30:39,622 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/snapshot-00000000685EAB0F-24F4DAEE.v1
02:30:39,623 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/snapshot-00000000685EAB0F-24F4DAEE.v1
02:30:39,623 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/TermInfo.properties
02:30:39,623 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/journal-v1-1.log
02:30:39,623 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/metadata-v1-1.log
02:30:39,623 PM [follower23-%d902783] [DEBUG] RaftStorage - follower23: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:39.620026606Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8793059607687493075/follower23/snapshot-00000000685EAB0F-24F4DAEE.v1, size=88, entryCompress=NONE, stateCompress=NONE}
02:30:39,623 PM [follower23-%d902783] [DEBUG] EntryStoreCompleter - follower23: 1 completion(s) pending
02:30:39,623 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] EntryStoreCompleter - follower23: running 1 completion(s)
02:30:39,623 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - follower23: snapshot is durable as of 2025-06-27T14:30:39.620026606Z
02:30:39,623 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower23: Snapshot success -  sequence number: -1
02:30:39,623 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PeerInfos - follower23: Updated server config: isVoting: false, peers: [PeerInfo [id=leader22, address=pekko://test/user/leader22, votingState=VOTING]]
02:30:39,624 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower23: support() called
02:30:39,624 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower23: applySnapshot called
02:30:39,624 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower23 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=3, followerId=follower23, chunkIndex=1, success=true}
02:30:39,624 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorSnapshotMessageSupport - follower23: SnapshotComplete received
02:30:39,625 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader22 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=3, followerId=follower23, chunkIndex=1, success=true}
02:30:39,625 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractLeader - leader22 (Leader): Snapshot successfully installed on follower follower23 (last chunk 1) - matchIndex set to 1, nextIndex set to 2
02:30:39,625 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - leader22: fakeSnapshot purging log to 0 for term 3
02:30:39,625 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorSnapshotMessageSupport - leader22: SnapshotComplete received
02:30:39,642 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithLessLeaderLogEntriesAfterNonPersistentLeaderRestart ending
02:30:39,642 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower23-collector25#1256676850]
02:30:39,653 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower23]
02:30:39,653 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - follower23: terminating thread VirtualThread[#334,follower23-writer-9]/waiting
02:30:39,654 PM [follower23-writer-9] [DEBUG] JournalWriteTask - follower23: received 1 entries in 1.219 s
02:30:39,654 PM [follower23-writer-9] [DEBUG] JournalWriteTask - follower23: persisted 1 entries in 52.80 μs
02:30:39,654 PM [follower23-writer-9] [DEBUG] JournalWriteTask - follower23: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@14cbfba8 stopped
02:30:39,654 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower23: journal closed
02:30:39,654 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower23: journal closed
02:30:39,654 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower23: stopped executor with 0 remaining tasks
02:30:39,654 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower23: stopped executor with 0 remaining tasks
02:30:39,655 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader22-collector26#-1076789722]
02:30:39,655 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader22]
02:30:39,655 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader22-collector27#119652569]
02:30:39,656 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader22]
02:30:39,656 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRaftStorage - leader22: terminating thread VirtualThread[#336,leader22-writer-11]/waiting
02:30:39,656 PM [leader22-writer-11] [DEBUG] JournalWriteTask - leader22: received 1 entries in 679.4 ms
02:30:39,656 PM [leader22-writer-11] [DEBUG] JournalWriteTask - leader22: persisted 1 entries in 22.94 μs
02:30:39,656 PM [leader22-writer-11] [DEBUG] JournalWriteTask - leader22: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@21917cd9 stopped
02:30:39,656 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - leader22: journal closed
02:30:39,656 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - leader22: journal closed
02:30:39,656 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader22: stopped executor with 0 remaining tasks
02:30:39,657 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader22: stopped executor with 0 remaining tasks
02:30:39,659 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithEmptyLeaderLogAfterNonPersistentLeaderRestart starting
02:30:39,664 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower29: Starting recovery with journal batch size 1000
02:30:39,664 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower29: started executor
02:30:39,664 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower29: started executor
02:30:39,665 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower29: metadata journal open: replayFrom=1 applyTo=0
02:30:39,665 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] EntryJournalV1 - follower29: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:39,666 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRaftStorage - follower29: journal open: applyTo=0
02:30:39,666 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower29] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:39,666 PM [follower29-writer-12] [DEBUG] JournalWriteTask - follower29: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1268c948 started
02:30:39,666 PM [follower29-writer-12] [DEBUG] JournalWriteTask - follower29: waiting for more work
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/TermInfo.properties does not exist
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/journal-v1-1.log
[WARN] [06/27/2025 14:30:39.668] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower29]: org.opendaylight.controller.cluster.common.actor.Monitor@5ad28bd2
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/metadata-v1-1.log
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftStorage - follower29: no eligible files found
[INFO] [pekkoDeadLetter][06/27/2025 14:30:39.665] [test-pekko.actor.default-dispatcher-8] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower29] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower29: Set currentTerm=1, votedFor=leader28
[WARN] [06/27/2025 14:30:39.668] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader28]: org.opendaylight.controller.cluster.common.actor.Monitor@1183307c
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PekkoRecovery - follower29: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:39,668 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PeerInfos - follower29: Updated server config: isVoting: false, peers: [PeerInfo [id=leader28, address=pekko://test/user/leader28, votingState=VOTING]]
02:30:39,669 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - follower29: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:39,669 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActor - leader28: Starting recovery with journal batch size 1000
02:30:39,670 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: started executor
02:30:39,670 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: started executor
02:30:39,670 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader28: metadata journal open: replayFrom=1 applyTo=0
02:30:39,671 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader28: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:39,671 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRaftStorage - leader28: journal open: applyTo=0
02:30:39,671 PM [leader28-writer-13] [DEBUG] JournalWriteTask - leader28: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@44a42211 started
02:30:39,672 PM [leader28-writer-13] [DEBUG] JournalWriteTask - leader28: waiting for more work
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/TermInfo.properties does not exist
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/journal-v1-1.log
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/metadata-v1-1.log
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader28: no eligible files found
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=1, votedFor=leader28
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRecovery - leader28: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:39,673 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PeerInfos - leader28: Updated server config: isVoting: true, peers: [PeerInfo [id=follower29, address=pekko://test/user/follower29, votingState=NON_VOTING]]
02:30:39,674 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader28: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:39,675 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] PropertiesTermInfoStore - follower29: Set currentTerm=1, votedFor=leader28
02:30:39,675 PM [test-pekko.actor.default-dispatcher-7] [INFO] PekkoRecovery - follower29: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader28}
02:30:39,675 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower29: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@74738a37
02:30:39,677 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=1, votedFor=leader28
02:30:39,677 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader28: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader28}
02:30:39,677 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader28: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@73ccc65f
02:30:40,104 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:40,104 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader28 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:40,104 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader28 (Follower): Received ElectionTimeout - switching to Candidate
02:30:40,105 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader28 (Candidate): Election: Candidate has following voting peers: []
02:30:40,223 PM [main] [ERROR] RaftActorTestKit - FindLeader failed
java.util.concurrent.TimeoutException: Future timed out after [100 milliseconds]
	at scala.library@2.13.16/scala.concurrent.impl.Promise$DefaultPromise.tryAwait0(Promise.scala:248)
	at scala.library@2.13.16/scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:261)
	at scala.library@2.13.16/scala.concurrent.Await$.$anonfun$result$1(package.scala:201)
	at scala.library@2.13.16/scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:62)
	at scala.library@2.13.16/scala.concurrent.Await$.result(package.scala:124)
	at scala.library@2.13.16/scala.concurrent.Await.result(package.scala)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActorTestKit.waitUntilLeader(RaftActorTestKit.java:59)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.AbstractRaftActorIntegrationTest.waitUntilLeader(AbstractRaftActorIntegrationTest.java:269)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest.setupLeaderAndNonVotingFollower(NonVotingFollowerIntegrationTest.java:442)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest.testFollowerResyncWithEmptyLeaderLogAfterNonPersistentLeaderRestart(NonVotingFollowerIntegrationTest.java:51)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
	at org.junit.vintage.engine.execution.VintageExecutor.executeClassesAndMethodsSequentially(VintageExecutor.java:93)
	at org.junit.vintage.engine.execution.VintageExecutor.executeAllChildren(VintageExecutor.java:73)
	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:70)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.executeEngine(EngineExecutionOrchestrator.java:230)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.failOrExecuteEngine(EngineExecutionOrchestrator.java:204)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:172)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:101)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:64)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:150)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:63)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:109)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:91)
	at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
	at org.junit.platform.launcher.core.InterceptingLauncher.lambda$execute$1(InterceptingLauncher.java:39)
	at org.junit.platform.launcher.core.ClasspathAlignmentCheckingLauncherInterceptor.intercept(ClasspathAlignmentCheckingLauncherInterceptor.java:25)
	at org.junit.platform.launcher.core.InterceptingLauncher.execute(InterceptingLauncher.java:38)
	at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
	at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:194)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
02:30:40,306 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=2, votedFor=leader28
02:30:40,306 PM [test-pekko.actor.default-dispatcher-17] [INFO] Candidate - leader28 (Candidate): Starting new election term 2
02:30:40,306 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader28 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
02:30:40,306 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader28 (Candidate): Received ElectionTimeout
02:30:40,306 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Election: Leader has following peers: [follower29]
02:30:40,307 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:40,307 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader28 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
02:30:40,307 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower29 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term
[WARN] [06/27/2025 14:30:40.308] [test-pekko.actor.default-dispatcher-17] [pekko://test/deadLetters] received dead letter from TestActor[pekko://test/user/leader28]: org.opendaylight.controller.cluster.raft.client.messages.FindLeaderReply@79e08130
02:30:40,314 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower29: Set currentTerm=2, votedFor=null
02:30:40,314 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower29: Last sync leader does not match current leader leader28, need to catch up to -1
02:30:40,314 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader28 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower29 updated: matchIndex: -1, nextIndex: 0
02:30:40,314 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/27/2025 14:30:40.314] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower29] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower29, initialSyncDone=false]
02:30:40,358 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader28: Persist data index=0 term=2 command=zero
02:30:40,358 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader28: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=true} 
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader28: 1 completion(s) pending
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader28 (Leader): Setting last applied to 0
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader28: Moving last applied index from -1 to 0
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader28: Applying state for log index 0 data zero
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader28: applyState called: zero
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:40,359 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): sendAppendEntries: 0 is present for follower follower29
02:30:40,360 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Sending AppendEntries to follower follower29: AppendEntries{term=2, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=true}]}
02:30:40,360 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader28: running 1 completion(s)
02:30:40,360 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader28: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=true}
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader28: Persist data index=1 term=2 command=one
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader28: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=true} 
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader28: 1 completion(s) pending
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader28 (Leader): Setting last applied to 1
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader28: Moving last applied index from 0 to 1
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader28: Applying state for log index 1 data one
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader28: applyState called: one
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:40,361 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): sendAppendEntries: 0 is present for follower follower29
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Sending AppendEntries to follower follower29: AppendEntries{term=2, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=true}]}
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader28: running 1 completion(s)
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader28: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=true}
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}]}
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Number of entries to be appended = 1
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Append entry to log zero
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower29: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=false} 
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower29: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=false}
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Log size is now 1
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Commit index set to 0
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower29, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower29: Lagging 0 entries behind leader leader28 and reached 0 (of expected -1)
02:30:40,362 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower29 (Follower): Setting last applied to 0
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower29: Moving last applied index from -1 to 0
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower29: Applying state for log index 0 data zero
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower29: applyState called: zero
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Number of entries to be appended = 2
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): matchEntry JournaledLogEntry{index=0, term=2, command=zero, pending=false} is present: existingEntryTerm: 2
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): After cleanup, lastIndex: 0, entries to be added from: 1
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Append entry to log one
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower29: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=false} 
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower29: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=false}
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Log size is now 2
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Commit index set to 1
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower29, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:40,363 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower29: Lagging 0 entries behind leader leader28 and reached 1 (of expected -1)
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower29 updated: matchIndex: 0, nextIndex: 1
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - leader28: fakeSnapshot purging log to 0 for term 2
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): sendAppendEntries: 1 is present for follower follower29
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Sending AppendEntries to follower follower29: AppendEntries{term=2, leaderId=leader28, prevLogIndex=0, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower29 updated: matchIndex: 1, nextIndex: 2
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader28: Persist data index=2 term=2 command=two
02:30:40,364 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader28: Append log entry and persist JournaledLogEntry{index=2, term=2, command=two, pending=true} 
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader28: 1 completion(s) pending
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - leader28 (Leader): Setting last applied to 2
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - leader28: Moving last applied index from 1 to 2
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader28: Applying state for log index 2 data two
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - leader28: applyState called: two
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): sendAppendEntries: 2 is present for follower follower29
02:30:40,365 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Sending AppendEntries to follower follower29: AppendEntries{term=2, leaderId=leader28, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=2, command=two, pending=true}]}
02:30:40,365 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower29 (Follower): Setting last applied to 1
02:30:40,365 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower29: Moving last applied index from 0 to 1
02:30:40,365 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower29: Applying state for log index 1 data one
02:30:40,365 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower29: applyState called: one
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader28: running 1 completion(s)
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader28: persist complete JournaledLogEntry{index=2, term=2, command=two, pending=true}
02:30:40,365 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - leader28: fakeSnapshot purging log to 1 for term 2
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader28, prevLogIndex=0, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Number of entries to be appended = 1
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): matchEntry JournaledLogEntry{index=1, term=2, command=one, pending=false} is present: existingEntryTerm: 2
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): After cleanup, lastIndex: 1, entries to be added from: 1
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Log size is now 2
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower29, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower29: Lagging 0 entries behind leader leader28 and reached 1 (of expected -1)
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - follower29: fakeSnapshot purging log to 0 for term 2
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader28, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:40,366 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Number of entries to be appended = 1
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Append entry to log two
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower29: Append log entry and persist JournaledLogEntry{index=2, term=2, command=two, pending=false} 
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower29: persist complete JournaledLogEntry{index=2, term=2, command=two, pending=false}
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Log size is now 2
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): Commit index set to 2
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower29, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower29: Lagging 0 entries behind leader leader28 and reached 2 (of expected -1)
02:30:40,367 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader28 (Leader): sendAppendEntries: 2 is present for follower follower29
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower29 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower29 (Follower): Setting last applied to 2
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower29: Moving last applied index from 1 to 2
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower29: Applying state for log index 2 data two
02:30:40,367 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower29: applyState called: two
02:30:40,367 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader28 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower29 updated: matchIndex: 2, nextIndex: 3
02:30:40,367 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
[WARN] [06/27/2025 14:30:40.368] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower29] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower29, initialSyncDone=true]
02:30:40,420 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRaftStorage - leader28: terminating thread VirtualThread[#344,leader28-writer-13]/waiting
02:30:40,421 PM [leader28-writer-13] [DEBUG] JournalWriteTask - leader28: received 1 entries in 749.1 ms
02:30:40,421 PM [leader28-writer-13] [DEBUG] JournalWriteTask - leader28: persisted 1 entries in 54.65 μs
02:30:40,421 PM [leader28-writer-13] [DEBUG] JournalWriteTask - leader28: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@44a42211 stopped
02:30:40,421 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader28: journal closed
02:30:40,421 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRaftStorage - leader28: journal closed
02:30:40,421 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: stopped executor with 0 remaining tasks
02:30:40,421 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: stopped executor with 0 remaining tasks
[WARN] [06/27/2025 14:30:40.424] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader28]: org.opendaylight.controller.cluster.common.actor.Monitor@530bea3f
02:30:40,424 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActor - leader28: Starting recovery with journal batch size 1000
02:30:40,424 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: started executor
02:30:40,425 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: started executor
02:30:40,425 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader28: metadata journal open: replayFrom=1 applyTo=0
02:30:40,425 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader28: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:40,425 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRaftStorage - leader28: journal open: applyTo=0
02:30:40,426 PM [leader28-writer-14] [DEBUG] JournalWriteTask - leader28: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@627ecfe8 started
02:30:40,426 PM [leader28-writer-14] [DEBUG] JournalWriteTask - leader28: waiting for more work
02:30:40,427 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=2, votedFor=leader28
02:30:40,427 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/TermInfo.properties
02:30:40,427 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/journal-v1-1.log
02:30:40,427 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/metadata-v1-1.log
02:30:40,427 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader28: no eligible files found
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=1, votedFor=leader28
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - leader28: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PeerInfos - leader28: Updated server config: isVoting: true, peers: [PeerInfo [id=follower29, address=pekko://test/user/follower29, votingState=NON_VOTING]]
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - leader28: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - leader28: restoring local TermInfo{term=2, votedFor=leader28}
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=2, votedFor=leader28
02:30:40,428 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader28: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2a9b9abc
02:30:40,854 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader28 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:40,854 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - leader28 (Follower): Received ElectionTimeout - switching to Candidate
02:30:40,854 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader28 (Candidate): Election: Candidate has following voting peers: []
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=3, votedFor=leader28
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [INFO] Candidate - leader28 (Candidate): Starting new election term 3
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader28 (Follower) :- Switching from behavior Follower to Candidate, election term: 3
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Candidate - leader28 (Candidate): Received ElectionTimeout
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Election: Leader has following peers: [follower29]
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:40,860 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActorBehavior - leader28 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3
02:30:40,861 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower29 (Follower): Term 3 in "AppendEntries{term=3, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term
02:30:40,865 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower29: Set currentTerm=3, votedFor=null
02:30:40,865 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower29: Lagging -3 entries behind leader leader28 and reached 2 (of expected -1)
02:30:40,865 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractLeader - leader28 (Leader): handleAppendEntriesReply: follower follower29 lastIndex 2 is ahead of our lastIndex -1 (snapshotIndex -1, snapshotTerm -1) - forcing install snaphot
02:30:40,865 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader28: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead.
02:30:40,866 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader28: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower29
02:30:40,866 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader28: lastSequenceNumber prior to capture: -1
02:30:40,866 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader28: support() called
02:30:40,867 PM [leader28-%d354893] [DEBUG] EntryStoreCompleter - leader28: 1 completion(s) pending
02:30:40,867 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader28: running 1 completion(s)
02:30:40,867 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader28: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1]
02:30:40,867 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader28: support() called
02:30:40,868 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/snapshot-00000000685EAB10-33C0954D.tmp
02:30:40,868 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader28: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1
02:30:40,868 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): sendInstallSnapshot
02:30:40,868 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] LeaderInstallSnapshotState - leader28 (Leader): Snapshot 157 bytes, total chunks to send: 1
02:30:40,868 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] LeaderInstallSnapshotState - leader28 (Leader): Next chunk: total length=157, offset=0, size=157, hashCode=1546036366
02:30:40,868 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): next snapshot chunk size for follower follower29: 157
02:30:40,869 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): InstallSnapshot sent to follower Vector(user, follower29), Chunk: 1/1
02:30:40,869 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower29 (Follower): handleInstallSnapshot: InstallSnapshot{term=3, leaderId=leader28, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=157, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=follower29, isVoting=false], ServerInfo[peerId=leader28, isVoting=true]]}}
02:30:40,869 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower29: Lagging -3 entries behind leader leader28 and reached 2 (of expected -1)
02:30:40,869 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotTracker - follower29 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
02:30:40,869 PM [test-pekko.actor.default-dispatcher-8] [INFO] Follower - follower29 (Follower): Snapshot received from leader: leader28
02:30:40,869 PM [test-pekko.actor.default-dispatcher-8] [INFO] SnapshotManager - follower29: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=157}}
02:30:40,870 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower29: support() called
02:30:40,870 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - follower29: Converted InstallSnapshot from leader: leader28 to state
02:30:40,871 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - follower29: lastSequenceNumber prior to persisting applied snapshot: -1
02:30:40,871 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower29: support() called
02:30:40,871 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/snapshot-00000000685EAB10-33F0B086.tmp
02:30:40,871 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/snapshot-00000000685EAB10-33C0954D.v1
02:30:40,871 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/snapshot-00000000685EAB10-33C0954D.v1
02:30:40,872 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/TermInfo.properties
02:30:40,872 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/journal-v1-1.log
02:30:40,872 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/metadata-v1-1.log
02:30:40,872 PM [leader28-%d354894] [DEBUG] RaftStorage - leader28: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:40.868259149Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/leader28/snapshot-00000000685EAB10-33C0954D.v1, size=88, entryCompress=NONE, stateCompress=NONE}
02:30:40,872 PM [leader28-%d354894] [DEBUG] EntryStoreCompleter - leader28: 1 completion(s) pending
02:30:40,872 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader28: running 1 completion(s)
02:30:40,872 PM [test-pekko.actor.default-dispatcher-8] [INFO] SnapshotManager - leader28: snapshot is durable as of 2025-06-27T14:30:40.868259149Z
02:30:40,872 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - leader28: Snapshot success -  sequence number: -1
02:30:40,872 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorSnapshotMessageSupport - leader28: SnapshotComplete received
02:30:40,875 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/snapshot-00000000685EAB10-33F0B086.v1
02:30:40,875 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/snapshot-00000000685EAB10-33F0B086.v1
02:30:40,875 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/TermInfo.properties
02:30:40,875 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/journal-v1-1.log
02:30:40,875 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/metadata-v1-1.log
02:30:40,875 PM [follower29-%d86611] [DEBUG] RaftStorage - follower29: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:40.871411846Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2683940818351357503/follower29/snapshot-00000000685EAB10-33F0B086.v1, size=88, entryCompress=NONE, stateCompress=NONE}
02:30:40,875 PM [follower29-%d86611] [DEBUG] EntryStoreCompleter - follower29: 1 completion(s) pending
02:30:40,875 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - follower29: running 1 completion(s)
02:30:40,875 PM [test-pekko.actor.default-dispatcher-8] [INFO] SnapshotManager - follower29: snapshot is durable as of 2025-06-27T14:30:40.871411846Z
02:30:40,876 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - follower29: Snapshot success -  sequence number: -1
02:30:40,876 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PeerInfos - follower29: Updated server config: isVoting: false, peers: [PeerInfo [id=leader28, address=pekko://test/user/leader28, votingState=VOTING]]
02:30:40,876 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower29: support() called
02:30:40,876 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower29: applySnapshot called
02:30:40,876 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower29 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=3, followerId=follower29, chunkIndex=1, success=true}
02:30:40,876 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorSnapshotMessageSupport - follower29: SnapshotComplete received
02:30:40,876 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=3, followerId=follower29, chunkIndex=1, success=true}
02:30:40,876 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractLeader - leader28 (Leader): Snapshot successfully installed on follower follower29 (last chunk 1) - matchIndex set to -1, nextIndex set to 0
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader28: Persist data index=0 term=3 command=zero-1
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader28: Append log entry and persist JournaledLogEntry{index=0, term=3, command=zero-1, pending=true} 
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader28: 1 completion(s) pending
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Replicate message: identifier: MockIdentifier{value=zero-1}, logIndex: 0, isSendImmediate: true
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - leader28 (Leader): Setting last applied to 0
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - leader28: Moving last applied index from -1 to 0
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader28: Applying state for log index 0 data zero-1
02:30:40,882 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - leader28: applyState called: zero-1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): sendAppendEntries: 0 is present for follower follower29
02:30:40,883 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader28 (Leader): Sending AppendEntries to follower follower29: AppendEntries{term=3, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}]}
02:30:40,883 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader28: running 1 completion(s)
02:30:40,883 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader28: persist complete JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader28, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}]}
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): Number of entries to be appended = 1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): Append entry to log zero-1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower29: Append log entry and persist JournaledLogEntry{index=0, term=3, command=zero-1, pending=false} 
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - follower29: persist complete JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): Log size is now 1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): Commit index set to 0
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower29, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower29: Lagging 0 entries behind leader leader28 and reached 0 (of expected -1)
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower29 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - follower29 (Follower): Setting last applied to 0
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - follower29: Moving last applied index from -1 to 0
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower29: Applying state for log index 0 data zero-1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower29: applyState called: zero-1
02:30:40,883 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower29 updated: matchIndex: 0, nextIndex: 1
02:30:40,884 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader28 (Leader): Checking sendAppendEntries for follower follower29: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:40,932 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithEmptyLeaderLogAfterNonPersistentLeaderRestart ending
02:30:40,933 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower29-collector31#-1887593516]
02:30:40,944 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower29]
02:30:40,944 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - follower29: terminating thread VirtualThread[#340,follower29-writer-12]/waiting
02:30:40,944 PM [follower29-writer-12] [DEBUG] JournalWriteTask - follower29: received 1 entries in 1.279 s
02:30:40,945 PM [follower29-writer-12] [DEBUG] JournalWriteTask - follower29: persisted 1 entries in 76.91 μs
02:30:40,945 PM [follower29-writer-12] [DEBUG] JournalWriteTask - follower29: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1268c948 stopped
02:30:40,945 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower29: journal closed
02:30:40,945 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower29: journal closed
02:30:40,945 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower29: stopped executor with 0 remaining tasks
02:30:40,945 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower29: stopped executor with 0 remaining tasks
02:30:40,945 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader28-collector32#1594537240]
02:30:40,946 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader28]
02:30:40,946 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader28-collector33#1769264269]
02:30:40,947 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader28]
02:30:40,947 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - leader28: terminating thread VirtualThread[#345,leader28-writer-14]/waiting
02:30:40,947 PM [leader28-writer-14] [DEBUG] JournalWriteTask - leader28: received 1 entries in 521.4 ms
02:30:40,947 PM [leader28-writer-14] [DEBUG] JournalWriteTask - leader28: persisted 1 entries in 32.60 μs
02:30:40,947 PM [leader28-writer-14] [DEBUG] JournalWriteTask - leader28: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@627ecfe8 stopped
02:30:40,947 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader28: journal closed
02:30:40,947 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader28: journal closed
02:30:40,947 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader28: stopped executor with 0 remaining tasks
02:30:40,947 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader28: stopped executor with 0 remaining tasks
02:30:40,950 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithOneMoreLeaderLogEntryAfterNonPersistentLeaderRestart starting
02:30:40,955 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower35: Starting recovery with journal batch size 1000
02:30:40,955 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower35: started executor
02:30:40,956 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower35: started executor
02:30:40,956 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower35: metadata journal open: replayFrom=1 applyTo=0
02:30:40,957 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower35: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:40,957 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower35: journal open: applyTo=0
02:30:40,957 PM [follower35-writer-15] [DEBUG] JournalWriteTask - follower35: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@7468860a started
02:30:40,957 PM [follower35-writer-15] [DEBUG] JournalWriteTask - follower35: waiting for more work
[WARN] [06/27/2025 14:30:40.957] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower35]: org.opendaylight.controller.cluster.common.actor.Monitor@1dfba1f3
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/TermInfo.properties does not exist
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower35: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/journal-v1-1.log
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower35: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/metadata-v1-1.log
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower35: no eligible files found
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower35: Set currentTerm=1, votedFor=leader34
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRecovery - follower35: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PeerInfos - follower35: Updated server config: isVoting: false, peers: [PeerInfo [id=leader34, address=pekko://test/user/leader34, votingState=VOTING]]
02:30:40,959 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower35: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:40,963 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower35: Set currentTerm=1, votedFor=leader34
02:30:40,963 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower35: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader34}
02:30:40,963 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower35: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@30f39ed4
[WARN] [06/27/2025 14:30:41.055] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader34]: org.opendaylight.controller.cluster.common.actor.Monitor@53391b4b
02:30:41,56 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActor - leader34: Starting recovery with journal batch size 1000
02:30:41,56 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader34: started executor
02:30:41,56 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader34: started executor
02:30:41,57 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader34: metadata journal open: replayFrom=1 applyTo=0
02:30:41,57 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader34: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:41,57 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRaftStorage - leader34: journal open: applyTo=0
02:30:41,58 PM [leader34-writer-16] [DEBUG] JournalWriteTask - leader34: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6d7841ea started
02:30:41,58 PM [leader34-writer-16] [DEBUG] JournalWriteTask - leader34: waiting for more work
02:30:41,59 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/TermInfo.properties does not exist
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/journal-v1-1.log
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/metadata-v1-1.log
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader34: no eligible files found
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=1, votedFor=leader34
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - leader34: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PeerInfos - leader34: Updated server config: isVoting: true, peers: [PeerInfo [id=follower35, address=pekko://test/user/follower35, votingState=NON_VOTING]]
02:30:41,60 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - leader34: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:41,65 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=1, votedFor=leader34
02:30:41,65 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - leader34: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader34}
02:30:41,65 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader34: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@55f8cdcc
02:30:41,544 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:41,544 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - leader34 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:41,544 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - leader34 (Follower): Received ElectionTimeout - switching to Candidate
02:30:41,544 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Candidate - leader34 (Candidate): Election: Candidate has following voting peers: []
02:30:41,548 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=2, votedFor=leader34
02:30:41,548 PM [test-pekko.actor.default-dispatcher-8] [INFO] Candidate - leader34 (Candidate): Starting new election term 2
02:30:41,548 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActorBehavior - leader34 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
02:30:41,548 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Candidate - leader34 (Candidate): Received ElectionTimeout
02:30:41,549 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader34 (Leader): Election: Leader has following peers: [follower35]
02:30:41,549 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:41,549 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActorBehavior - leader34 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
02:30:41,550 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower35 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term
02:30:41,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower35: Set currentTerm=2, votedFor=null
02:30:41,554 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower35 updated: matchIndex: -1, nextIndex: 0
02:30:41,555 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:41,555 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower35: Last sync leader does not match current leader leader34, need to catch up to -1
[WARN] [06/27/2025 14:30:41.555] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower35] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower35, initialSyncDone=false]
02:30:41,570 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader34: Persist data index=0 term=2 command=zero
02:30:41,570 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader34: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=true} 
02:30:41,570 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:41,570 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader34 (Leader): Setting last applied to 0
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader34: Moving last applied index from -1 to 0
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader34: Applying state for log index 0 data zero
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader34: applyState called: zero
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): sendAppendEntries: 0 is present for follower follower35
02:30:41,571 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Sending AppendEntries to follower follower35: AppendEntries{term=2, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=true}]}
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader34: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=true}
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}]}
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): Number of entries to be appended = 1
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): Append entry to log zero
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader34: Persist data index=1 term=2 command=one
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower35: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=false} 
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader34: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=true} 
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower35: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=false}
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): Log size is now 1
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): Commit index set to 0
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower35, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower35: Lagging 0 entries behind leader leader34 and reached 0 (of expected -1)
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - leader34 (Leader): Setting last applied to 1
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower35 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - leader34: Moving last applied index from 0 to 1
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - follower35 (Follower): Setting last applied to 0
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader34: Applying state for log index 1 data one
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - follower35: Moving last applied index from -1 to 0
02:30:41,572 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - leader34: applyState called: one
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - follower35: Applying state for log index 0 data zero
02:30:41,572 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower35: applyState called: zero
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): sendAppendEntries: 0 is present for follower follower35
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Sending AppendEntries to follower follower35: AppendEntries{term=2, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=true}]}
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - leader34: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=true}
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): Number of entries to be appended = 2
[WARN] [06/27/2025 14:30:41.573] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower35] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower35, initialSyncDone=true]
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): matchEntry JournaledLogEntry{index=0, term=2, command=zero, pending=false} is present: existingEntryTerm: 2
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower35 updated: matchIndex: 0, nextIndex: 1
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): After cleanup, lastIndex: 0, entries to be added from: 1
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): Append entry to log one
02:30:41,573 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader34: fakeSnapshot purging log to 0 for term 2
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower35: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=false} 
02:30:41,573 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] ReplicatedLogImpl - follower35: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=false}
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): Log size is now 2
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): Commit index set to 1
02:30:41,574 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0
02:30:41,574 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): sendAppendEntries: 1 is present for follower follower35
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower35, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower35: Lagging 0 entries behind leader leader34 and reached 1 (of expected -1)
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorBehavior - follower35 (Follower): Setting last applied to 1
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractReplicatedLog - follower35: Moving last applied index from 0 to 1
02:30:41,574 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Sending AppendEntries to follower follower35: AppendEntries{term=2, leaderId=leader34, prevLogIndex=0, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActor - follower35: Applying state for log index 1 data one
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - follower35: applyState called: one
02:30:41,574 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower35 updated: matchIndex: 1, nextIndex: 2
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader34, prevLogIndex=0, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): Number of entries to be appended = 1
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): matchEntry JournaledLogEntry{index=1, term=2, command=one, pending=false} is present: existingEntryTerm: 2
02:30:41,574 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): After cleanup, lastIndex: 1, entries to be added from: 1
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): Log size is now 2
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] Follower - follower35 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower35, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SyncStatusTracker - follower35: Lagging 0 entries behind leader leader34 and reached 1 (of expected -1)
02:30:41,574 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - follower35: fakeSnapshot purging log to 0 for term 2
02:30:41,632 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - leader34: terminating thread VirtualThread[#350,leader34-writer-16]/waiting
02:30:41,632 PM [leader34-writer-16] [DEBUG] JournalWriteTask - leader34: received 1 entries in 574.6 ms
02:30:41,632 PM [leader34-writer-16] [DEBUG] JournalWriteTask - leader34: persisted 1 entries in 51.82 μs
02:30:41,633 PM [leader34-writer-16] [DEBUG] JournalWriteTask - leader34: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6d7841ea stopped
02:30:41,633 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader34: journal closed
02:30:41,633 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader34: journal closed
02:30:41,633 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader34: stopped executor with 0 remaining tasks
02:30:41,633 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader34: stopped executor with 0 remaining tasks
[WARN] [06/27/2025 14:30:41.635] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader34]: org.opendaylight.controller.cluster.common.actor.Monitor@2da94331
02:30:41,638 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - leader34: Starting recovery with journal batch size 1000
02:30:41,639 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader34: started executor
02:30:41,639 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader34: started executor
02:30:41,643 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader34: metadata journal open: replayFrom=1 applyTo=0
02:30:41,644 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader34: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:41,644 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader34: journal open: applyTo=0
02:30:41,644 PM [leader34-writer-17] [DEBUG] JournalWriteTask - leader34: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@12b2ad6e started
02:30:41,644 PM [leader34-writer-17] [DEBUG] JournalWriteTask - leader34: waiting for more work
02:30:41,647 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=2, votedFor=leader34
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/TermInfo.properties
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/journal-v1-1.log
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/metadata-v1-1.log
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader34: no eligible files found
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=1, votedFor=leader34
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - leader34: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PeerInfos - leader34: Updated server config: isVoting: true, peers: [PeerInfo [id=follower35, address=pekko://test/user/follower35, votingState=NON_VOTING]]
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader34: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - leader34: restoring local TermInfo{term=2, votedFor=leader34}
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=2, votedFor=leader34
02:30:41,648 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader34: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2dc31c0b
02:30:42,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - leader34 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:42,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - leader34 (Follower): Received ElectionTimeout - switching to Candidate
02:30:42,95 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Candidate - leader34 (Candidate): Election: Candidate has following voting peers: []
02:30:42,100 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - leader34: Set currentTerm=3, votedFor=leader34
02:30:42,101 PM [test-pekko.actor.default-dispatcher-18] [INFO] Candidate - leader34 (Candidate): Starting new election term 3
02:30:42,101 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActorBehavior - leader34 (Follower) :- Switching from behavior Follower to Candidate, election term: 3
02:30:42,101 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Candidate - leader34 (Candidate): Received ElectionTimeout
02:30:42,101 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Election: Leader has following peers: [follower35]
02:30:42,102 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:42,102 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActorBehavior - leader34 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader34: Persist data index=0 term=3 command=zero-1
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader34: Append log entry and persist JournaledLogEntry{index=0, term=3, command=zero-1, pending=true} 
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Replicate message: identifier: MockIdentifier{value=zero-1}, logIndex: 0, isSendImmediate: true
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader34 (Leader): Setting last applied to 0
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader34: Moving last applied index from -1 to 0
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader34: Applying state for log index 0 data zero-1
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader34: applyState called: zero-1
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:42,106 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader34: persist complete JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader34: Persist data index=1 term=3 command=one-1
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader34: Append log entry and persist JournaledLogEntry{index=1, term=3, command=one-1, pending=true} 
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Replicate message: identifier: MockIdentifier{value=one-1}, logIndex: 1, isSendImmediate: true
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader34 (Leader): Setting last applied to 1
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader34: Moving last applied index from 0 to 1
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader34: Applying state for log index 1 data one-1
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader34: applyState called: one-1
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:42,107 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader34: persist complete JournaledLogEntry{index=1, term=3, command=one-1, pending=true}
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader34: Persist data index=2 term=3 command=two-1
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader34: Append log entry and persist JournaledLogEntry{index=2, term=3, command=two-1, pending=true} 
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Replicate message: identifier: MockIdentifier{value=two-1}, logIndex: 2, isSendImmediate: true
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader34 (Leader): Setting last applied to 2
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader34: Moving last applied index from 1 to 2
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader34: Applying state for log index 2 data two-1
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader34: applyState called: two-1
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:42,108 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader34: persist complete JournaledLogEntry{index=2, term=3, command=two-1, pending=true}
02:30:42,215 PM [test-pekko.actor.default-dispatcher-18] [INFO] Follower - follower35 (Follower): Term 3 in "AppendEntries{term=3, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term
02:30:42,220 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower35: Set currentTerm=3, votedFor=null
02:30:42,220 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower35: Lagging -2 entries behind leader leader34 and reached 1 (of expected -1)
02:30:42,220 PM [test-pekko.actor.default-dispatcher-5] [INFO] AbstractLeader - leader34 (Leader): handleAppendEntriesReply: follower follower35 last log term 2 for index 1 conflicts with the leader's 3 - set the follower's next index to 0
02:30:42,220 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader34 (Leader): Checking sendAppendEntries for follower follower35: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1
02:30:42,220 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader34 (Leader): sendAppendEntries: 0 is present for follower follower35
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader34 (Leader): Sending AppendEntries to follower follower35: AppendEntries{term=3, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}, JournaledLogEntry{index=1, term=3, command=one-1, pending=false}, JournaledLogEntry{index=2, term=3, command=two-1, pending=false}]}
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower35 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader34, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}, JournaledLogEntry{index=1, term=3, command=one-1, pending=false}, JournaledLogEntry{index=2, term=3, command=two-1, pending=false}]}
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower35 (Follower): Number of entries to be appended = 3
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower35 (Follower): matchEntry JournaledLogEntry{index=0, term=3, command=zero-1, pending=false} is present: existingEntryTerm: 2
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower35 (Follower): Removing entries from log starting at 0, commitIndex: 1, lastApplied: 1
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower35 (Follower): Could not remove entries - sending reply to force snapshot
02:30:42,221 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower35: Lagging 1 entries behind leader leader34 and reached 1 (of expected -1)
02:30:42,222 PM [test-pekko.actor.default-dispatcher-7] [INFO] AbstractLeader - leader34 (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=follower35, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=true, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1
02:30:42,222 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader34: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=3, lastIndex=2, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower35
02:30:42,222 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader34: lastSequenceNumber prior to capture: -1
02:30:42,222 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader34: support() called
02:30:42,223 PM [leader34-%d736460] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:42,224 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:42,224 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader34: Persising snapshot at EntryInfo[index=2, term=3]/EntryInfo[index=2, term=3]
02:30:42,224 PM [test-pekko.actor.default-dispatcher-7] [INFO] MockRaftActor - leader34: support() called
02:30:42,224 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader34: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1
02:30:42,224 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader34 (Leader): sendInstallSnapshot
02:30:42,224 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/snapshot-00000000685EAB12-0D63C8C3.tmp
02:30:42,225 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] LeaderInstallSnapshotState - leader34 (Leader): Snapshot 316 bytes, total chunks to send: 1
02:30:42,225 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] LeaderInstallSnapshotState - leader34 (Leader): Next chunk: total length=316, offset=0, size=316, hashCode=-635913009
02:30:42,225 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader34 (Leader): next snapshot chunk size for follower follower35: 316
02:30:42,225 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] AbstractLeader - leader34 (Leader): InstallSnapshot sent to follower Vector(user, follower35), Chunk: 1/1
02:30:42,225 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower35 (Follower): handleInstallSnapshot: InstallSnapshot{term=3, leaderId=leader34, lastIncludedIndex=2, lastIncludedTerm=3, datasize=316, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=follower35, isVoting=false], ServerInfo[peerId=leader34, isVoting=true]]}}
02:30:42,225 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower35: Lagging 1 entries behind leader leader34 and reached 1 (of expected -1)
02:30:42,226 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotTracker - follower35 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
02:30:42,226 PM [test-pekko.actor.default-dispatcher-18] [INFO] Follower - follower35 (Follower): Snapshot received from leader: leader34
02:30:42,226 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - follower35: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=316}}
02:30:42,226 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower35: support() called
02:30:42,227 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/snapshot-00000000685EAB12-0D63C8C3.v1
02:30:42,227 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/snapshot-00000000685EAB12-0D63C8C3.v1
02:30:42,227 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/TermInfo.properties
02:30:42,227 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/journal-v1-1.log
02:30:42,227 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/metadata-v1-1.log
02:30:42,227 PM [leader34-%d736461] [DEBUG] RaftStorage - leader34: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:42.224643267Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/leader34/snapshot-00000000685EAB12-0D63C8C3.v1, size=88, entryCompress=NONE, stateCompress=NONE}
02:30:42,228 PM [leader34-%d736461] [DEBUG] EntryStoreCompleter - leader34: 1 completion(s) pending
02:30:42,228 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower35: Converted InstallSnapshot from leader: leader34 to state
02:30:42,228 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower35: lastSequenceNumber prior to persisting applied snapshot: -1
02:30:42,228 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower35: support() called
02:30:42,229 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] EntryStoreCompleter - leader34: running 1 completion(s)
02:30:42,229 PM [test-pekko.actor.default-dispatcher-7] [INFO] SnapshotManager - leader34: snapshot is durable as of 2025-06-27T14:30:42.224643267Z
02:30:42,229 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] SnapshotManager - leader34: Snapshot success -  sequence number: -1
02:30:42,229 PM [test-pekko.actor.default-dispatcher-7] [DEBUG] RaftActorSnapshotMessageSupport - leader34: SnapshotComplete received
02:30:42,229 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/snapshot-00000000685EAB12-0DA96F22.tmp
02:30:42,231 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/snapshot-00000000685EAB12-0DA96F22.v1
02:30:42,231 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/snapshot-00000000685EAB12-0DA96F22.v1
02:30:42,231 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/TermInfo.properties
02:30:42,231 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/journal-v1-1.log
02:30:42,231 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/metadata-v1-1.log
02:30:42,231 PM [follower35-%d859578] [DEBUG] RaftStorage - follower35: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:42.229207842Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11639374712186129533/follower35/snapshot-00000000685EAB12-0DA96F22.v1, size=88, entryCompress=NONE, stateCompress=NONE}
02:30:42,231 PM [follower35-%d859578] [DEBUG] EntryStoreCompleter - follower35: 1 completion(s) pending
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryStoreCompleter - follower35: running 1 completion(s)
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [INFO] SnapshotManager - follower35: snapshot is durable as of 2025-06-27T14:30:42.229207842Z
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SnapshotManager - follower35: Snapshot success -  sequence number: -1
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PeerInfos - follower35: Updated server config: isVoting: false, peers: [PeerInfo [id=leader34, address=pekko://test/user/leader34, votingState=VOTING]]
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower35: support() called
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [INFO] MockRaftActor - follower35: applySnapshot called
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower35 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=3, followerId=follower35, chunkIndex=1, success=true}
02:30:42,232 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorSnapshotMessageSupport - follower35: SnapshotComplete received
02:30:42,232 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader34 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=3, followerId=follower35, chunkIndex=1, success=true}
02:30:42,233 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractLeader - leader34 (Leader): Snapshot successfully installed on follower follower35 (last chunk 1) - matchIndex set to 2, nextIndex set to 3
02:30:42,233 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader34: fakeSnapshot purging log to 1 for term 3
02:30:42,260 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithOneMoreLeaderLogEntryAfterNonPersistentLeaderRestart ending
02:30:42,260 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower35-collector37#950582273]
02:30:42,272 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower35]
02:30:42,272 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower35: terminating thread VirtualThread[#349,follower35-writer-15]/waiting
02:30:42,272 PM [follower35-writer-15] [DEBUG] JournalWriteTask - follower35: received 1 entries in 1.315 s
02:30:42,273 PM [follower35-writer-15] [DEBUG] JournalWriteTask - follower35: persisted 1 entries in 54.73 μs
02:30:42,273 PM [follower35-writer-15] [DEBUG] JournalWriteTask - follower35: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@7468860a stopped
02:30:42,273 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower35: journal closed
02:30:42,273 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower35: journal closed
02:30:42,273 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower35: stopped executor with 0 remaining tasks
02:30:42,273 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower35: stopped executor with 0 remaining tasks
02:30:42,274 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader34-collector38#-494518915]
02:30:42,274 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader34]
02:30:42,275 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader34-collector39#581052322]
02:30:42,276 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader34]
02:30:42,276 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader34: terminating thread VirtualThread[#351,leader34-writer-17]/waiting
02:30:42,276 PM [leader34-writer-17] [DEBUG] JournalWriteTask - leader34: received 1 entries in 632.4 ms
02:30:42,277 PM [leader34-writer-17] [DEBUG] JournalWriteTask - leader34: persisted 1 entries in 33.78 μs
02:30:42,277 PM [leader34-writer-17] [DEBUG] JournalWriteTask - leader34: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@12b2ad6e stopped
02:30:42,277 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader34: journal closed
02:30:42,277 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader34: journal closed
02:30:42,277 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader34: stopped executor with 0 remaining tasks
02:30:42,277 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader34: stopped executor with 0 remaining tasks
02:30:42,279 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithMoreLeaderLogEntriesAndDownPeerAfterNonPersistentLeaderRestart starting
[WARN] [06/27/2025 14:30:42.283] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower41]: org.opendaylight.controller.cluster.common.actor.Monitor@5d0015f0
02:30:42,284 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - follower41: Starting recovery with journal batch size 1000
02:30:42,285 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower41: started executor
02:30:42,285 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower41: started executor
02:30:42,285 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower41: metadata journal open: replayFrom=1 applyTo=0
02:30:42,287 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower41: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:42,287 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower41: journal open: applyTo=0
02:30:42,287 PM [follower41-writer-18] [DEBUG] JournalWriteTask - follower41: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6b99cf1f started
02:30:42,287 PM [follower41-writer-18] [DEBUG] JournalWriteTask - follower41: waiting for more work
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/TermInfo.properties does not exist
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower41: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/journal-v1-1.log
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower41: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/metadata-v1-1.log
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower41: no eligible files found
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower41: Set currentTerm=1, votedFor=leader40
[WARN] [06/27/2025 14:30:42.289] [test-pekko.actor.default-dispatcher-7] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader40]: org.opendaylight.controller.cluster.common.actor.Monitor@7f0184fd
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRecovery - follower41: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PeerInfos - follower41: Updated server config: isVoting: false, peers: [PeerInfo [id=leader40, address=pekko://test/user/leader40, votingState=VOTING]]
02:30:42,289 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower41: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:42,290 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActor - leader40: Starting recovery with journal batch size 1000
02:30:42,290 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader40: started executor
02:30:42,290 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader40: started executor
02:30:42,291 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader40: metadata journal open: replayFrom=1 applyTo=0
02:30:42,292 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryJournalV1 - leader40: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:42,292 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRaftStorage - leader40: journal open: applyTo=0
02:30:42,292 PM [leader40-writer-19] [DEBUG] JournalWriteTask - leader40: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@2e9b4c08 started
02:30:42,292 PM [leader40-writer-19] [DEBUG] JournalWriteTask - leader40: waiting for more work
02:30:42,293 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/TermInfo.properties does not exist
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/journal-v1-1.log
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/metadata-v1-1.log
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftStorage - leader40: no eligible files found
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=1, votedFor=leader40
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PekkoRecovery - leader40: Received ReplicatedLogEntry for recovery: index: 0, size: 141
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PeerInfos - leader40: Updated server config: isVoting: true, peers: [PeerInfo [id=follower41, address=pekko://test/user/follower41, votingState=NON_VOTING]]
02:30:42,294 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - leader40: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:42,294 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PropertiesTermInfoStore - follower41: Set currentTerm=1, votedFor=leader40
02:30:42,294 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRecovery - follower41: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader40}
02:30:42,294 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - follower41: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7fd5fa0d
02:30:42,297 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=1, votedFor=leader40
02:30:42,297 PM [test-pekko.actor.default-dispatcher-8] [INFO] PekkoRecovery - leader40: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader40}
02:30:42,297 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader40: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@5a15c99b
02:30:42,804 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:42,805 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - leader40 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:42,805 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - leader40 (Follower): Received ElectionTimeout - switching to Candidate
02:30:42,805 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Candidate - leader40 (Candidate): Election: Candidate has following voting peers: []
02:30:42,810 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=2, votedFor=leader40
02:30:42,810 PM [test-pekko.actor.default-dispatcher-8] [INFO] Candidate - leader40 (Candidate): Starting new election term 2
02:30:42,810 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActorBehavior - leader40 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
02:30:42,811 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Candidate - leader40 (Candidate): Received ElectionTimeout
02:30:42,811 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Election: Leader has following peers: [follower41]
02:30:42,811 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:42,811 PM [test-pekko.actor.default-dispatcher-8] [INFO] RaftActorBehavior - leader40 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
02:30:42,811 PM [test-pekko.actor.default-dispatcher-8] [INFO] Follower - follower41 (Follower): Term 2 in "AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term
02:30:42,814 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] PropertiesTermInfoStore - follower41: Set currentTerm=2, votedFor=null
02:30:42,814 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower41: Last sync leader does not match current leader leader40, need to catch up to -1
02:30:42,814 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower41 updated: matchIndex: -1, nextIndex: 0
02:30:42,814 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/27/2025 14:30:42.815] [test-pekko.actor.default-dispatcher-8] [pekko://test/user/follower41] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower41, initialSyncDone=false]
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader40: Persist data index=0 term=2 command=zero
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=true} 
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 0
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from -1 to 0
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader40: Applying state for log index 0 data zero
02:30:42,854 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - leader40: applyState called: zero
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 0 is present for follower follower41
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=true}]}
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=true}
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader40: Persist data index=1 term=2 command=one
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=true} 
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 1
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from 0 to 1
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader40: Applying state for log index 1 data one
02:30:42,855 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - leader40: applyState called: one
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 0 is present for follower follower41
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=true}]}
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=true}
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}]}
02:30:42,856 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Number of entries to be appended = 1
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Persist data index=2 term=2 command=two
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Append entry to log zero
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=2, term=2, command=two, pending=true} 
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower41: Append log entry and persist JournaledLogEntry{index=0, term=2, command=zero, pending=false} 
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower41: persist complete JournaledLogEntry{index=0, term=2, command=zero, pending=false}
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Log size is now 1
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 2
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Commit index set to 0
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from 1 to 2
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Applying state for log index 2 data two
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader40: applyState called: two
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower41, logLastIndex=0, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower41: Lagging 0 entries behind leader leader40 and reached 0 (of expected -1)
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - follower41 (Follower): Setting last applied to 0
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - follower41: Moving last applied index from -1 to 0
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - follower41: Applying state for log index 0 data zero
02:30:42,857 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower41: applyState called: zero
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1
02:30:42,857 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 0 is present for follower follower41
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=false}, JournaledLogEntry{index=2, term=2, command=two, pending=true}]}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=false}]}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Number of entries to be appended = 2
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=0, term=2, command=zero, pending=false} is present: existingEntryTerm: 2
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=2, term=2, command=two, pending=true}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): After cleanup, lastIndex: 0, entries to be added from: 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Append entry to log one
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower41: Append log entry and persist JournaledLogEntry{index=1, term=2, command=one, pending=false} 
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower41: persist complete JournaledLogEntry{index=1, term=2, command=one, pending=false}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower41 updated: matchIndex: 0, nextIndex: 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Log size is now 2
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader40: fakeSnapshot purging log to 0 for term 2
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Commit index set to 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower41, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: 0
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 1 is present for follower follower41
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower41: Lagging 0 entries behind leader leader40 and reached 1 (of expected -1)
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=2, leaderId=leader40, prevLogIndex=0, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}, JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - follower41 (Follower): Setting last applied to 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - follower41: Moving last applied index from 0 to 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - follower41: Applying state for log index 1 data one
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower41: applyState called: one
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower41 updated: matchIndex: 1, nextIndex: 2
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader40: fakeSnapshot purging log to 1 for term 2
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 2 is present for follower follower41
02:30:42,858 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=2, leaderId=leader40, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=2, command=zero, pending=false}, JournaledLogEntry{index=1, term=2, command=one, pending=false}, JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Number of entries to be appended = 3
02:30:42,858 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=0, term=2, command=zero, pending=false} is present: existingEntryTerm: 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=1, term=2, command=one, pending=false} is present: existingEntryTerm: 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): After cleanup, lastIndex: 1, entries to be added from: 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Append entry to log two
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower41: Append log entry and persist JournaledLogEntry{index=2, term=2, command=two, pending=false} 
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower41: persist complete JournaledLogEntry{index=2, term=2, command=two, pending=false}
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Log size is now 3
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Commit index set to 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower41, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:42,859 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower41 updated: matchIndex: 2, nextIndex: 3
02:30:42,859 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower41: Lagging 0 entries behind leader leader40 and reached 2 (of expected -1)
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - follower41 (Follower): Setting last applied to 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - follower41: Moving last applied index from 1 to 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - follower41: Applying state for log index 2 data two
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower41: applyState called: two
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader40, prevLogIndex=0, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=0, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=2, command=one, pending=false}, JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Number of entries to be appended = 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=1, term=2, command=one, pending=false} is present: existingEntryTerm: 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=2, term=2, command=two, pending=false} is present: existingEntryTerm: 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): After cleanup, lastIndex: 2, entries to be added from: 2
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): Log size is now 3
02:30:42,859 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower41 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower41, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:42,860 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower41: Lagging 0 entries behind leader leader40 and reached 2 (of expected -1)
02:30:42,860 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SnapshotManager - follower41: fakeSnapshot purging log to 0 for term 2
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=2, leaderId=leader40, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=2, command=two, pending=false}]}
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): Number of entries to be appended = 1
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=2, term=2, command=two, pending=false} is present: existingEntryTerm: 2
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): After cleanup, lastIndex: 2, entries to be added from: 1
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): Log size is now 2
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): handleAppendEntries returning : AppendEntriesReply{term=2, success=true, followerId=follower41, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower41: Lagging 0 entries behind leader leader40 and reached 2 (of expected -1)
02:30:42,860 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower41: fakeSnapshot purging log to 1 for term 2
[WARN] [06/27/2025 14:30:42.860] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/follower41] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower41, initialSyncDone=true]
02:30:43,45 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower41: Lagging 0 entries behind leader leader40 and reached 2 (of expected -1)
02:30:43,69 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PekkoRaftStorage - leader40: terminating thread VirtualThread[#356,leader40-writer-19]/waiting
02:30:43,70 PM [leader40-writer-19] [DEBUG] JournalWriteTask - leader40: received 1 entries in 777.7 ms
02:30:43,70 PM [leader40-writer-19] [DEBUG] JournalWriteTask - leader40: persisted 1 entries in 47.08 μs
02:30:43,70 PM [leader40-writer-19] [DEBUG] JournalWriteTask - leader40: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@2e9b4c08 stopped
02:30:43,70 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - leader40: journal closed
02:30:43,70 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - leader40: journal closed
02:30:43,70 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader40: stopped executor with 0 remaining tasks
02:30:43,70 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - leader40: stopped executor with 0 remaining tasks
[WARN] [06/27/2025 14:30:43.073] [test-pekko.actor.default-dispatcher-8] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower42]: org.opendaylight.controller.cluster.common.actor.Monitor@6a76a3e9
02:30:43,74 PM [test-pekko.actor.default-dispatcher-18] [INFO] RaftActor - follower42: Starting recovery with journal batch size 1000
02:30:43,74 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower42: started executor
02:30:43,75 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftStorage - follower42: started executor
02:30:43,75 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower42: metadata journal open: replayFrom=1 applyTo=0
02:30:43,76 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryJournalV1 - follower42: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:43,76 PM [test-pekko.actor.default-dispatcher-18] [INFO] PekkoRaftStorage - follower42: journal open: applyTo=0
02:30:43,77 PM [follower42-writer-20] [DEBUG] JournalWriteTask - follower42: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@4235d768 started
02:30:43,77 PM [follower42-writer-20] [DEBUG] JournalWriteTask - follower42: waiting for more work
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower42/TermInfo.properties does not exist
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower42: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower42/journal-v1-1.log
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower42: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower42/metadata-v1-1.log
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - follower42: no eligible files found
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRecovery - follower42: Received ReplicatedLogEntry for recovery: index: 0, size: 171
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PeerInfos - follower42: Updated server config: isVoting: true, peers: [PeerInfo [id=downPeer, address=null, votingState=NON_VOTING], PeerInfo [id=follower41, address=pekko://test/user/follower41, votingState=NON_VOTING], PeerInfo [id=leader40, address=pekko://test/user/leader40, votingState=VOTING]]
02:30:43,79 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower42: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:43,83 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - follower42: Set currentTerm=0, votedFor=null
02:30:43,83 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - follower42: Local TermInfo store seeded with TermInfo{term=0}
02:30:43,83 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower42: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@49caeb53
[WARN] [06/27/2025 14:30:43.085] [test-pekko.actor.default-dispatcher-18] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader40]: org.opendaylight.controller.cluster.common.actor.Monitor@406516da
02:30:43,86 PM [test-pekko.actor.default-dispatcher-17] [INFO] RaftActor - leader40: Starting recovery with journal batch size 1000
02:30:43,86 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader40: started executor
02:30:43,86 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader40: started executor
02:30:43,87 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader40: metadata journal open: replayFrom=1 applyTo=0
02:30:43,87 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryJournalV1 - leader40: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0
02:30:43,87 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRaftStorage - leader40: journal open: applyTo=0
02:30:43,87 PM [leader40-writer-21] [DEBUG] JournalWriteTask - leader40: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1fff86c3 started
02:30:43,87 PM [leader40-writer-21] [DEBUG] JournalWriteTask - leader40: waiting for more work
02:30:43,88 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=2, votedFor=leader40
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/TermInfo.properties
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/journal-v1-1.log
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/metadata-v1-1.log
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftStorage - leader40: no eligible files found
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=2, votedFor=leader40
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRecovery - leader40: Received ReplicatedLogEntry for recovery: index: 0, size: 171
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PeerInfos - leader40: Updated server config: isVoting: true, peers: [PeerInfo [id=follower42, address=pekko://test/user/follower42, votingState=VOTING], PeerInfo [id=downPeer, address=null, votingState=NON_VOTING], PeerInfo [id=follower41, address=pekko://test/user/follower41, votingState=NON_VOTING]]
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [INFO] PekkoRecovery - leader40: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PekkoRecovery - leader40: restoring local TermInfo{term=2, votedFor=leader40}
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=2, votedFor=leader40
02:30:43,89 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - leader40: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2177de14
02:30:43,525 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader40 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.170.8:17355, Removed)
02:30:43,525 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader40 (Follower): Received ElectionTimeout - switching to Candidate
02:30:43,525 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader40 (Candidate): Election: Candidate has following voting peers: [follower42]
02:30:43,531 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader40: Set currentTerm=3, votedFor=leader40
02:30:43,531 PM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader40 (Candidate): Starting new election term 3
02:30:43,531 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader40 (Candidate): Sending RequestVote{term=3, candidateId=leader40, lastLogIndex=-1, lastLogTerm=-1} to peer follower42
02:30:43,531 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader40 (Follower) :- Switching from behavior Follower to Candidate, election term: 3
02:30:43,531 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower42 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
02:30:43,532 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower42 (Follower): Cluster state: []
02:30:43,532 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower42 (Follower): Candidate in requestVote:RequestVote{term=3, candidateId=leader40, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
02:30:43,532 PM [test-pekko.actor.default-dispatcher-5] [INFO] Follower - follower42 (Follower): Term 3 in "RequestVote{term=3, candidateId=leader40, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
02:30:43,537 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower42: Set currentTerm=3, votedFor=null
02:30:43,537 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower42 (Follower): In requestVote: RequestVote{term=3, candidateId=leader40, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1
02:30:43,540 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower42: Set currentTerm=3, votedFor=leader40
02:30:43,540 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActorBehavior - follower42 (Follower): requestVote returning: RequestVoteReply{term=3, voteGranted=true}
02:30:43,540 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader40 (Candidate): RaftRPC message received RequestVoteReply{term=3, voteGranted=true}, my term is 3
02:30:43,540 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader40 (Candidate): handleRequestVoteReply: RequestVoteReply{term=3, voteGranted=true}, current voteCount: 1
02:30:43,541 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader40 (Leader): Election: Leader has following peers: [follower42, downPeer, follower41]
02:30:43,541 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:43,541 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
02:30:43,541 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader40 (Candidate) :- Switching from behavior Candidate to Leader, election term: 3
02:30:43,541 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower42: Last sync leader does not match current leader leader40, need to catch up to -1
02:30:43,542 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower42 updated: matchIndex: -1, nextIndex: 0
02:30:43,542 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/27/2025 14:30:43.542] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower42] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower42, initialSyncDone=false]
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Persist data index=0 term=3 command=zero-1
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=0, term=3, command=zero-1, pending=true} 
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=zero-1}, logIndex: 0, isSendImmediate: true
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 0 is present for follower follower42
02:30:43,544 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower42: AppendEntries{term=3, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}]}
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower42 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}]}
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower42 (Follower): Number of entries to be appended = 1
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower42 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower42 (Follower): Append entry to log zero-1
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower42: Append log entry and persist JournaledLogEntry{index=0, term=3, command=zero-1, pending=true} 
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower42: persist complete JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower42 (Follower): Log size is now 1
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - follower42 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower42, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:43,545 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached -1 (of expected -1)
02:30:43,545 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
[WARN] [06/27/2025 14:30:43.545] [test-pekko.actor.default-dispatcher-17] [pekko://test/user/follower42] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower42, initialSyncDone=true]
02:30:43,545 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=0, term=3, command=zero-1, pending=true}
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower42 updated: matchIndex: 0, nextIndex: 1
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 0
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from -1 to 0
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Applying state for log index 0 data zero-1
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader40: applyState called: zero-1
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Persist data index=1 term=3 command=one-1
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=1, term=3, command=one-1, pending=true} 
02:30:43,546 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=one-1}, logIndex: 1, isSendImmediate: true
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): Commit index set to 0
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached 0 (of expected -1)
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 1 is present for follower follower42
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - follower42 (Follower): Setting last applied to 0
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - follower42: Moving last applied index from -1 to 0
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - follower42: Applying state for log index 0 data zero-1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - follower42: applyState called: zero-1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower42: AppendEntries{term=3, leaderId=leader40, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=3, command=one-1, pending=true}]}
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=1, term=3, command=one-1, pending=true}
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader40, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=3, command=one-1, pending=false}]}
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): Number of entries to be appended = 1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 1 is present for follower follower42
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): Append entry to log one-1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower42: Append log entry and persist JournaledLogEntry{index=1, term=3, command=one-1, pending=false} 
02:30:43,547 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - follower42: persist complete JournaledLogEntry{index=1, term=3, command=one-1, pending=false}
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Persist data index=2 term=3 command=two-1
02:30:43,547 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=2, term=3, command=two-1, pending=true} 
02:30:43,548 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): Log size is now 2
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,548 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] Follower - follower42 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower42, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=two-1}, logIndex: 2, isSendImmediate: true
02:30:43,548 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached 0 (of expected -1)
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 1 is present for follower follower42
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=2, term=3, command=two-1, pending=true}
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower42 updated: matchIndex: 1, nextIndex: 2
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from 0 to 1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Applying state for log index 1 data one-1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader40: applyState called: one-1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1
02:30:43,548 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 2 is present for follower follower42
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower42: AppendEntries{term=3, leaderId=leader40, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=3, command=two-1, pending=false}]}
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Persist data index=3 term=3 command=three-1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=3, term=3, command=three-1, pending=true} 
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader40, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=3, command=two-1, pending=false}]}
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Number of entries to be appended = 1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=three-1}, logIndex: 3, isSendImmediate: true
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Append entry to log two-1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower42: Append log entry and persist JournaledLogEntry{index=2, term=3, command=two-1, pending=false} 
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower42: persist complete JournaledLogEntry{index=2, term=3, command=two-1, pending=false}
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 2 is present for follower follower42
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Log size is now 3
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Commit index set to 1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower42, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=3, term=3, command=three-1, pending=true}
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActor - leader40: Persist data index=4 term=3 command=four-1
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] ReplicatedLogImpl - leader40: Append log entry and persist JournaledLogEntry{index=4, term=3, command=four-1, pending=true} 
02:30:43,549 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached 1 (of expected -1)
02:30:43,549 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,550 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 2
02:30:43,550 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Replicate message: identifier: MockIdentifier{value=four-1}, logIndex: 4, isSendImmediate: true
02:30:43,550 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower42 (Follower): Setting last applied to 1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower42: Moving last applied index from 0 to 1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower42: Applying state for log index 1 data one-1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower42: applyState called: one-1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: -1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 2 is present for follower follower42
02:30:43,550 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] ReplicatedLogImpl - leader40: persist complete JournaledLogEntry{index=4, term=3, command=four-1, pending=true}
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower42 updated: matchIndex: 2, nextIndex: 3
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 2
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from 1 to 2
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] RaftActor - leader40: Applying state for log index 2 data two-1
02:30:43,550 PM [test-pekko.actor.default-dispatcher-8] [INFO] MockRaftActor - leader40: applyState called: two-1
02:30:43,551 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: -1
02:30:43,551 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 3 is present for follower follower42
02:30:43,551 PM [test-pekko.actor.default-dispatcher-8] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower42: AppendEntries{term=3, leaderId=leader40, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=3, command=three-1, pending=false}, JournaledLogEntry{index=4, term=3, command=four-1, pending=false}]}
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader40, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=3, command=three-1, pending=false}, JournaledLogEntry{index=4, term=3, command=four-1, pending=false}]}
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Number of entries to be appended = 2
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Append entry to log three-1
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower42: Append log entry and persist JournaledLogEntry{index=3, term=3, command=three-1, pending=false} 
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower42: persist complete JournaledLogEntry{index=3, term=3, command=three-1, pending=false}
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Append entry to log four-1
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower42: Append log entry and persist JournaledLogEntry{index=4, term=3, command=four-1, pending=false} 
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] ReplicatedLogImpl - follower42: persist complete JournaledLogEntry{index=4, term=3, command=four-1, pending=false}
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Log size is now 5
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Commit index set to 2
02:30:43,551 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): handleAppendEntries returning : AppendEntriesReply{term=3, success=true, followerId=follower42, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:43,551 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower42 updated: matchIndex: 4, nextIndex: 5
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader40 (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 3
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from 2 to 3
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader40: Applying state for log index 3 data three-1
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader40: applyState called: three-1
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader40 (Leader): Setting last applied to 4
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader40: Moving last applied index from 3 to 4
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader40: Applying state for log index 4 data four-1
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader40: applyState called: four-1
02:30:43,552 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower42: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: -1
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached 2 (of expected -1)
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 4
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower42 (Follower): Setting last applied to 2
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower42: Moving last applied index from 1 to 2
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower42: Applying state for log index 2 data two-1
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower42: applyState called: two-1
02:30:43,553 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): Commit index set to 4
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached 4 (of expected -1)
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower42 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower42 (Follower): Setting last applied to 3
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower42: Moving last applied index from 2 to 3
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower42: Applying state for log index 3 data three-1
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower42: applyState called: three-1
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorBehavior - follower42 (Follower): Setting last applied to 4
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractReplicatedLog - follower42: Moving last applied index from 3 to 4
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActor - follower42: Applying state for log index 4 data four-1
02:30:43,554 PM [test-pekko.actor.default-dispatcher-17] [INFO] MockRaftActor - follower42: applyState called: four-1
02:30:43,655 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower41 (Follower): Term 3 in "AppendEntries{term=3, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term
02:30:43,655 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower42: Lagging 0 entries behind leader leader40 and reached 4 (of expected -1)
02:30:43,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower41: Set currentTerm=3, votedFor=null
02:30:43,658 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower41: Lagging -3 entries behind leader leader40 and reached 2 (of expected -1)
02:30:43,659 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractLeader - leader40 (Leader): handleAppendEntriesReply: follower follower41 last log term 2 for index 2 conflicts with the leader's 3 - set the follower's next index to 1
02:30:43,659 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 1, leaderLastIndex: 4, leaderSnapShotIndex: -1
02:30:43,659 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 1 is present for follower follower41
02:30:43,659 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=3, leaderId=leader40, prevLogIndex=0, prevLogTerm=3, leaderCommit=4, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=3, command=one-1, pending=false}, JournaledLogEntry{index=2, term=3, command=two-1, pending=false}, JournaledLogEntry{index=3, term=3, command=three-1, pending=false}, JournaledLogEntry{index=4, term=3, command=four-1, pending=false}]}
02:30:43,659 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader40, prevLogIndex=0, prevLogTerm=3, leaderCommit=4, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=3, command=one-1, pending=false}, JournaledLogEntry{index=2, term=3, command=two-1, pending=false}, JournaledLogEntry{index=3, term=3, command=three-1, pending=false}, JournaledLogEntry{index=4, term=3, command=four-1, pending=false}]}
02:30:43,659 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower41 (Follower): The prevLogIndex 0 was found in the log but the term 2 is not equal to the append entries prevLogTerm 3 - lastIndex: 2, snapshotIndex: 1, snapshotTerm: 2
02:30:43,659 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower41 (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=follower41, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
02:30:43,659 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower41: Lagging 2 entries behind leader leader40 and reached 2 (of expected -1)
02:30:43,659 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=follower41, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractLeader - leader40 (Leader): follower follower41 last log term 2 conflicts with the leader's 3 - dec next index to 0
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Checking sendAppendEntries for follower follower41: active: true, followerNextIndex: 0, leaderLastIndex: 4, leaderSnapShotIndex: -1
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendAppendEntries: 0 is present for follower follower41
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): Sending AppendEntries to follower follower41: AppendEntries{term=3, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=4, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}, JournaledLogEntry{index=1, term=3, command=one-1, pending=false}, JournaledLogEntry{index=2, term=3, command=two-1, pending=false}, JournaledLogEntry{index=3, term=3, command=three-1, pending=false}, JournaledLogEntry{index=4, term=3, command=four-1, pending=false}]}
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower41 (Follower): handleAppendEntries: AppendEntries{term=3, leaderId=leader40, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=4, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=3, command=zero-1, pending=false}, JournaledLogEntry{index=1, term=3, command=one-1, pending=false}, JournaledLogEntry{index=2, term=3, command=two-1, pending=false}, JournaledLogEntry{index=3, term=3, command=three-1, pending=false}, JournaledLogEntry{index=4, term=3, command=four-1, pending=false}]}
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower41 (Follower): Number of entries to be appended = 5
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=0, term=3, command=zero-1, pending=false} is present: existingEntryTerm: -1
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] Follower - follower41 (Follower): matchEntry JournaledLogEntry{index=1, term=3, command=one-1, pending=false} is present: existingEntryTerm: 2
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower41 (Follower): Removing entries from log starting at 1, commitIndex: 2, lastApplied: 2
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [INFO] Follower - follower41 (Follower): Could not remove entries - sending reply to force snapshot
02:30:43,660 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SyncStatusTracker - follower41: Lagging 2 entries behind leader leader40 and reached 2 (of expected -1)
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [INFO] AbstractLeader - leader40 (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=follower41, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=true, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader40: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=4, lastAppliedTerm=3, lastIndex=4, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower41
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - leader40: lastSequenceNumber prior to capture: -1
02:30:43,660 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader40: support() called
02:30:43,662 PM [leader40-%d177539] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,662 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,662 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader40: Persising snapshot at EntryInfo[index=4, term=3]/EntryInfo[index=4, term=3]
02:30:43,662 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - leader40: support() called
02:30:43,662 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - leader40: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1
02:30:43,662 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): sendInstallSnapshot
02:30:43,662 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/snapshot-00000000685EAB13-2780EB31.tmp
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] LeaderInstallSnapshotState - leader40 (Leader): Snapshot 355 bytes, total chunks to send: 1
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] LeaderInstallSnapshotState - leader40 (Leader): Next chunk: total length=355, offset=0, size=355, hashCode=951985329
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): next snapshot chunk size for follower follower41: 355
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] AbstractLeader - leader40 (Leader): InstallSnapshot sent to follower Vector(user, follower41), Chunk: 1/1
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): handleInstallSnapshot: InstallSnapshot{term=3, leaderId=leader40, lastIncludedIndex=4, lastIncludedTerm=3, datasize=355, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=follower42, isVoting=true], ServerInfo[peerId=downPeer, isVoting=false], ServerInfo[peerId=follower41, isVoting=false], ServerInfo[peerId=leader40, isVoting=true]]}}
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SyncStatusTracker - follower41: Lagging 2 entries behind leader leader40 and reached 2 (of expected -1)
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotTracker - follower41 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [INFO] Follower - follower41 (Follower): Snapshot received from leader: leader40
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - follower41: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=355}}
02:30:43,663 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower41: support() called
02:30:43,664 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower41: Converted InstallSnapshot from leader: leader40 to state
02:30:43,664 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower41: lastSequenceNumber prior to persisting applied snapshot: -1
02:30:43,664 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower41: support() called
02:30:43,665 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/snapshot-00000000685EAB13-27A45D35.tmp
02:30:43,667 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/snapshot-00000000685EAB13-2780EB31.v1
02:30:43,667 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/snapshot-00000000685EAB13-2780EB31.v1
02:30:43,667 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/TermInfo.properties
02:30:43,667 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/journal-v1-1.log
02:30:43,667 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/metadata-v1-1.log
02:30:43,667 PM [leader40-%d177540] [DEBUG] RaftStorage - leader40: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:43.662760241Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/leader40/snapshot-00000000685EAB13-2780EB31.v1, size=112, entryCompress=NONE, stateCompress=NONE}
02:30:43,667 PM [leader40-%d177540] [DEBUG] EntryStoreCompleter - leader40: 1 completion(s) pending
02:30:43,668 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] EntryStoreCompleter - leader40: running 1 completion(s)
02:30:43,668 PM [test-pekko.actor.default-dispatcher-17] [INFO] SnapshotManager - leader40: snapshot is durable as of 2025-06-27T14:30:43.662760241Z
02:30:43,668 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] SnapshotManager - leader40: Snapshot success -  sequence number: -1
02:30:43,668 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] RaftActorSnapshotMessageSupport - leader40: SnapshotComplete received
02:30:43,669 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/snapshot-00000000685EAB13-27A45D35.v1
02:30:43,670 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/snapshot-00000000685EAB13-27A45D35.v1
02:30:43,670 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/TermInfo.properties
02:30:43,670 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/journal-v1-1.log
02:30:43,670 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/metadata-v1-1.log
02:30:43,670 PM [follower41-%d149748] [DEBUG] RaftStorage - follower41: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-06-27T14:30:43.665083189Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest13598223962760935842/follower41/snapshot-00000000685EAB13-27A45D35.v1, size=112, entryCompress=NONE, stateCompress=NONE}
02:30:43,670 PM [follower41-%d149748] [DEBUG] EntryStoreCompleter - follower41: 1 completion(s) pending
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] EntryStoreCompleter - follower41: running 1 completion(s)
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [INFO] SnapshotManager - follower41: snapshot is durable as of 2025-06-27T14:30:43.665083189Z
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] SnapshotManager - follower41: Snapshot success -  sequence number: -1
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] PeerInfos - follower41: Updated server config: isVoting: false, peers: [PeerInfo [id=follower42, address=null, votingState=VOTING], PeerInfo [id=downPeer, address=null, votingState=NON_VOTING], PeerInfo [id=leader40, address=pekko://test/user/leader40, votingState=VOTING]]
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower41: support() called
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [INFO] MockRaftActor - follower41: applySnapshot called
02:30:43,670 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] Follower - follower41 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=3, followerId=follower41, chunkIndex=1, success=true}
02:30:43,671 PM [test-pekko.actor.default-dispatcher-18] [DEBUG] RaftActorSnapshotMessageSupport - follower41: SnapshotComplete received
02:30:43,671 PM [test-pekko.actor.default-dispatcher-17] [DEBUG] AbstractLeader - leader40 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=3, followerId=follower41, chunkIndex=1, success=true}
02:30:43,671 PM [test-pekko.actor.default-dispatcher-17] [INFO] AbstractLeader - leader40 (Leader): Snapshot successfully installed on follower follower41 (last chunk 1) - matchIndex set to 4, nextIndex set to 5
02:30:43,695 PM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithMoreLeaderLogEntriesAndDownPeerAfterNonPersistentLeaderRestart ending
02:30:43,695 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower41-collector43#383993636]
02:30:43,707 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower41]
02:30:43,707 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - follower41: terminating thread VirtualThread[#355,follower41-writer-18]/waiting
02:30:43,707 PM [follower41-writer-18] [DEBUG] JournalWriteTask - follower41: received 1 entries in 1.420 s
02:30:43,707 PM [follower41-writer-18] [DEBUG] JournalWriteTask - follower41: persisted 1 entries in 55.67 μs
02:30:43,708 PM [follower41-writer-18] [DEBUG] JournalWriteTask - follower41: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@6b99cf1f stopped
02:30:43,708 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower41: journal closed
02:30:43,708 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - follower41: journal closed
02:30:43,708 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower41: stopped executor with 0 remaining tasks
02:30:43,708 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower41: stopped executor with 0 remaining tasks
02:30:43,708 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader40-collector44#-2084023916]
02:30:43,709 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader40]
02:30:43,709 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower42-collector45#-1564020605]
02:30:43,709 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower42]
02:30:43,709 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - follower42: terminating thread VirtualThread[#357,follower42-writer-20]/waiting
02:30:43,710 PM [follower42-writer-20] [DEBUG] JournalWriteTask - follower42: received 1 entries in 633.0 ms
02:30:43,710 PM [follower42-writer-20] [DEBUG] JournalWriteTask - follower42: persisted 1 entries in 33.47 μs
02:30:43,710 PM [follower42-writer-20] [DEBUG] JournalWriteTask - follower42: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@4235d768 stopped
02:30:43,710 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - follower42: journal closed
02:30:43,710 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - follower42: journal closed
02:30:43,710 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower42: stopped executor with 0 remaining tasks
02:30:43,710 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower42: stopped executor with 0 remaining tasks
02:30:43,711 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader40-collector46#837755194]
02:30:43,711 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader40]
02:30:43,711 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRaftStorage - leader40: terminating thread VirtualThread[#358,leader40-writer-21]/waiting
02:30:43,712 PM [leader40-writer-21] [DEBUG] JournalWriteTask - leader40: received 1 entries in 624.1 ms
02:30:43,712 PM [leader40-writer-21] [DEBUG] JournalWriteTask - leader40: persisted 1 entries in 33.51 μs
02:30:43,712 PM [leader40-writer-21] [DEBUG] JournalWriteTask - leader40: task org.opendaylight.controller.cluster.raft.spi.JournalWriteTask@1fff86c3 stopped
02:30:43,712 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] EntryJournalV1 - leader40: journal closed
02:30:43,712 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRaftStorage - leader40: journal closed
02:30:43,712 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader40: stopped executor with 0 remaining tasks
02:30:43,712 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader40: stopped executor with 0 remaining tasks
[INFO] [06/27/2025 14:30:43.714] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
02:30:43,714 PM [test-pekko.actor.default-dispatcher-16] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
02:30:43,717 PM [test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Exiting completed
02:30:43,717 PM [test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Shutting down...
[WARN] [06/27/2025 14:30:43.717] [test-pekko.actor.default-dispatcher-17] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.170.8:17355)
02:30:43,717 PM [test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://test@10.30.170.8:17355] - Successfully shut down
[INFO] [06/27/2025 14:30:43.716] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Exiting completed
[INFO] [06/27/2025 14:30:43.716] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Shutting down...
[INFO] [06/27/2025 14:30:43.716] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.170.8:17355] - Successfully shut down
[INFO] [06/27/2025 14:30:43.719] [test-pekko.remote.default-remote-dispatcher-10] [pekko://test@10.30.170.8:17355/system/remoting-terminator] Shutting down remote daemon.
[INFO] [06/27/2025 14:30:43.719] [test-pekko.remote.default-remote-dispatcher-10] [pekko://test@10.30.170.8:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports.
02:30:43,719 PM [test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.
02:30:43,719 PM [test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports.
[INFO] [06/27/2025 14:30:43.721] [test-pekko.remote.default-remote-dispatcher-10] [pekko://test@10.30.170.8:17355/system/remoting-terminator] Remoting shut down.
02:30:43,722 PM [test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down.
[DEBUG] [06/27/2025 14:30:43.722] [test-pekko.actor.internal-dispatcher-3] [EventStream] shutting down: StandardOutLogger