[DEBUG] [07/05/2025 09:19:47.980] [main] [EventStream(pekko://test)] logger log1-TestEventListener started 09:19:47,981 AM [test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started [DEBUG] [07/05/2025 09:19:47.981] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started [DEBUG] [07/05/2025 09:19:47.981] [main] [EventStream(pekko://test)] Default Loggers started [INFO] [07/05/2025 09:19:48.003] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.171.247:17355] with UID [6847667383170650205] 09:19:48,4 AM [test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.171.247:17355] with UID [6847667383170650205] [INFO] [07/05/2025 09:19:48.004] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - Starting up, Pekko version [1.0.3] ... 09:19:48,5 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - Starting up, Pekko version [1.0.3] ... [INFO] [07/05/2025 09:19:48.008] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - Registered cluster JMX MBean [pekko:type=Cluster] [INFO] [07/05/2025 09:19:48.008] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - Started up successfully 09:19:48,8 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - Registered cluster JMX MBean [pekko:type=Cluster] 09:19:48,8 AM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - Started up successfully [INFO] [07/05/2025 09:19:48.009] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:19:48,9 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:19:48,10 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining [INFO] [07/05/2025 09:19:48.010] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:19:48,26 AM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerLeaderStateChanges [WARN] [07/05/2025 09:19:48.034] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower18]: org.opendaylight.controller.cluster.common.actor.Monitor@3633400e 09:19:48,34 AM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower18] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][07/05/2025 09:19:48.034] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower18] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:48,38 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - follower18: Starting recovery with journal batch size 1000 09:19:48,38 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower18: started executor 09:19:48,38 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower18: started executor 09:19:48,40 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower18: metadata journal open: replayFrom=1 applyTo=0 09:19:48,40 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - follower18: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:19:48,41 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - follower18: journal open: applyTo=0 09:19:48,41 AM [follower18-writer-7] [DEBUG] JournalWriteTask - follower18: journal writer started 09:19:48,41 AM [follower18-writer-7] [DEBUG] JournalWriteTask - follower18: waiting to receive commands 09:19:48,45 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/TermInfo.properties does not exist 09:19:48,46 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/journal-v1-1.log 09:19:48,46 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/metadata-v1-1.log 09:19:48,46 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: no eligible files found 09:19:48,46 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower18: Set currentTerm=1, votedFor=leader17 09:19:48,46 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - follower18: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:19:48,46 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PeerInfos - follower18: Updated server config: isVoting: false, peers: [PeerInfo [id=leader17, address=pekko://test/user/leader17, votingState=VOTING]] 09:19:48,47 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower18: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [07/05/2025 09:19:48.049] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader17]: org.opendaylight.controller.cluster.common.actor.Monitor@126fcebc 09:19:48,49 AM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader17] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][07/05/2025 09:19:48.049] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader17] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:48,50 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower18: Set currentTerm=1, votedFor=leader17 09:19:48,50 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower18: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader17} 09:19:48,50 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower18: taking snapshot to clear Pekko persistence to 2 09:19:48,50 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/snapshot-000000006868EE34-0305488A.tmp 09:19:48,52 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - leader17: Starting recovery with journal batch size 1000 09:19:48,53 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader17: started executor 09:19:48,53 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader17: started executor 09:19:48,54 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader17: metadata journal open: replayFrom=1 applyTo=0 09:19:48,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/snapshot-000000006868EE34-0305488A.v1 09:19:48,54 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader17: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:19:48,54 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader17: journal open: applyTo=0 09:19:48,54 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/snapshot-000000006868EE34-0305488A.v1 09:19:48,54 AM [leader17-writer-8] [DEBUG] JournalWriteTask - leader17: journal writer started 09:19:48,54 AM [leader17-writer-8] [DEBUG] JournalWriteTask - leader17: waiting to receive commands 09:19:48,55 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/TermInfo.properties 09:19:48,55 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/journal-v1-1.log 09:19:48,55 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/metadata-v1-1.log 09:19:48,55 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower18: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:19:48.050677898Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/follower18/snapshot-000000006868EE34-0305488A.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:19:48,55 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower18: Pekko recovery completed and can restore from snapshot 09:19:48,55 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower18: no journal: skipping journal recovery 09:19:48,56 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower18: ignoring DeleteMessagesSuccess(2) 09:19:48,57 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/TermInfo.properties does not exist 09:19:48,57 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/journal-v1-1.log 09:19:48,57 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/metadata-v1-1.log 09:19:48,57 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: no eligible files found 09:19:48,57 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader17: Set currentTerm=1, votedFor=leader17 09:19:48,58 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PekkoRecovery - leader17: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:19:48,58 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PeerInfos - leader17: Updated server config: isVoting: true, peers: [PeerInfo [id=follower18, address=pekko://test/user/follower18, votingState=NON_VOTING]] 09:19:48,58 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader17: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:19:48,60 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader17: Set currentTerm=1, votedFor=leader17 09:19:48,60 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader17: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader17} 09:19:48,60 AM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - leader17: taking snapshot to clear Pekko persistence to 2 09:19:48,60 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/snapshot-000000006868EE34-039A0908.tmp 09:19:48,61 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/snapshot-000000006868EE34-039A0908.v1 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/snapshot-000000006868EE34-039A0908.v1 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/TermInfo.properties 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/journal-v1-1.log 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/metadata-v1-1.log 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader17: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:19:48.060426504Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest3600538753031965816/leader17/snapshot-000000006868EE34-039A0908.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader17: Pekko recovery completed and can restore from snapshot 09:19:48,62 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader17: no journal: skipping journal recovery 09:19:48,63 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader17: ignoring DeleteMessagesSuccess(2) 09:19:48,514 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:19:48,515 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader17 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.247:17355, Removed) 09:19:48,515 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - leader17 (Follower): Received ElectionTimeout - switching to Candidate 09:19:48,515 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader17 (Candidate): Election: Candidate has following voting peers: [] 09:19:48,519 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - leader17: Set currentTerm=2, votedFor=leader17 09:19:48,519 AM [test-pekko.actor.default-dispatcher-6] [INFO] Candidate - leader17 (Candidate): Starting new election term 2 09:19:48,519 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader17 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:19:48,519 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader17 (Candidate): Received ElectionTimeout 09:19:48,519 AM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader17 (Leader): Election: Leader has following peers: [follower18] 09:19:48,520 AM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader17 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:19:53,567 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/roleChangeNotifier20#-1753944375] 09:19:53,580 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower18-collector21#734487703] 09:19:53,582 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower18] 09:19:53,582 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - follower18: terminating thread VirtualThread[#334,follower18-writer-7]/waiting 09:19:53,583 AM [follower18-writer-7] [DEBUG] JournalWriteTask - follower18: received 1 commands after 5.542 s 09:19:53,583 AM [follower18-writer-7] [DEBUG] JournalWriteTask - follower18: completed 1 commands in 46.45 μs 09:19:53,583 AM [follower18-writer-7] [DEBUG] JournalWriteTask - follower18: journal writer stopped 09:19:53,584 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower18: journal closed 09:19:53,584 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower18: journal closed 09:19:53,584 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower18: stopped executor with 0 remaining tasks 09:19:53,584 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower18: stopped executor with 0 remaining tasks 09:19:53,585 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader17-collector22#1542606628] 09:19:53,585 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader17] 09:19:53,586 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - leader17: terminating thread VirtualThread[#336,leader17-writer-8]/waiting 09:19:53,586 AM [leader17-writer-8] [DEBUG] JournalWriteTask - leader17: received 1 commands after 5.531 s 09:19:53,586 AM [leader17-writer-8] [DEBUG] JournalWriteTask - leader17: completed 1 commands in 14.70 μs 09:19:53,586 AM [leader17-writer-8] [DEBUG] JournalWriteTask - leader17: journal writer stopped 09:19:53,586 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader17: journal closed 09:19:53,586 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader17: journal closed 09:19:53,586 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader17: stopped executor with 0 remaining tasks 09:19:53,586 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader17: stopped executor with 0 remaining tasks 09:19:53,591 AM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithLessLeaderLogEntriesAfterNonPersistentLeaderRestart starting 09:19:53,595 AM [test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower24] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [WARN] [07/05/2025 09:19:53.595] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower24]: org.opendaylight.controller.cluster.common.actor.Monitor@7ca0c7f5 [INFO] [pekkoDeadLetter][07/05/2025 09:19:53.595] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower24] to Actor[pekko://test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:53,597 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - follower24: Starting recovery with journal batch size 1000 09:19:53,597 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower24: started executor 09:19:53,597 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower24: started executor 09:19:53,598 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower24: metadata journal open: replayFrom=1 applyTo=0 09:19:53,598 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower24: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:19:53,598 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower24: journal open: applyTo=0 09:19:53,599 AM [follower24-writer-9] [DEBUG] JournalWriteTask - follower24: journal writer started 09:19:53,599 AM [follower24-writer-9] [DEBUG] JournalWriteTask - follower24: waiting to receive commands [WARN] [07/05/2025 09:19:53.601] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader23]: org.opendaylight.controller.cluster.common.actor.Monitor@67bff51c [INFO] [pekkoDeadLetter][07/05/2025 09:19:53.601] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader23] to Actor[pekko://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:53,602 AM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader23] to Actor[pekko://test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:53,602 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/TermInfo.properties does not exist 09:19:53,602 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/journal-v1-1.log 09:19:53,603 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/metadata-v1-1.log 09:19:53,603 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: no eligible files found 09:19:53,603 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower24: Set currentTerm=1, votedFor=leader23 09:19:53,603 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRecovery - follower24: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:19:53,603 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PeerInfos - follower24: Updated server config: isVoting: false, peers: [PeerInfo [id=leader23, address=pekko://test/user/leader23, votingState=VOTING]] 09:19:53,603 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - leader23: Starting recovery with journal batch size 1000 09:19:53,603 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower24: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:19:53,603 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: started executor 09:19:53,604 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: started executor 09:19:53,605 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader23: metadata journal open: replayFrom=1 applyTo=0 09:19:53,606 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader23: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:19:53,606 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader23: journal open: applyTo=0 09:19:53,606 AM [leader23-writer-10] [DEBUG] JournalWriteTask - leader23: journal writer started 09:19:53,606 AM [leader23-writer-10] [DEBUG] JournalWriteTask - leader23: waiting to receive commands 09:19:53,607 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - follower24: Set currentTerm=1, votedFor=leader23 09:19:53,607 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower24: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader23} 09:19:53,607 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - follower24: taking snapshot to clear Pekko persistence to 2 09:19:53,607 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/snapshot-000000006868EE39-243355B6.tmp 09:19:53,608 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/TermInfo.properties does not exist 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/journal-v1-1.log 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/metadata-v1-1.log 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: no eligible files found 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - leader23: Set currentTerm=1, votedFor=leader23 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRecovery - leader23: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PeerInfos - leader23: Updated server config: isVoting: true, peers: [PeerInfo [id=follower24, address=pekko://test/user/follower24, votingState=NON_VOTING]] 09:19:53,609 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader23: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:19:53,609 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/snapshot-000000006868EE39-243355B6.v1 09:19:53,609 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/snapshot-000000006868EE39-243355B6.v1 09:19:53,609 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/TermInfo.properties 09:19:53,610 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/journal-v1-1.log 09:19:53,610 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/metadata-v1-1.log 09:19:53,610 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - follower24: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:19:53.607344054Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/follower24/snapshot-000000006868EE39-243355B6.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:19:53,610 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower24: Pekko recovery completed and can restore from snapshot 09:19:53,610 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower24: no journal: skipping journal recovery 09:19:53,611 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - follower24: ignoring DeleteMessagesSuccess(2) 09:19:53,611 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - leader23: Set currentTerm=1, votedFor=leader23 09:19:53,611 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader23: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader23} 09:19:53,611 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader23: taking snapshot to clear Pekko persistence to 2 09:19:53,611 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/snapshot-000000006868EE39-24787EAA.tmp 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/snapshot-000000006868EE39-24787EAA.v1 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/snapshot-000000006868EE39-24787EAA.v1 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/TermInfo.properties 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/journal-v1-1.log 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/metadata-v1-1.log 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:19:53.611876522Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest318181477947199150/leader23/snapshot-000000006868EE39-24787EAA.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader23: Pekko recovery completed and can restore from snapshot 09:19:53,613 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader23: no journal: skipping journal recovery 09:19:53,614 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader23: ignoring DeleteMessagesSuccess(2) 09:19:54,34 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:19:54,35 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader23 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.247:17355, Removed) 09:19:54,35 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader23 (Follower): Received ElectionTimeout - switching to Candidate 09:19:54,35 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader23 (Candidate): Election: Candidate has following voting peers: [] 09:19:54,40 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader23: Set currentTerm=2, votedFor=leader23 09:19:54,40 AM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader23 (Candidate): Starting new election term 2 09:19:54,40 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader23 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:19:54,40 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader23 (Candidate): Received ElectionTimeout 09:19:54,41 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader23 (Leader): Election: Leader has following peers: [follower24] 09:19:54,41 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader23 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:19:59,100 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower24-collector26#-1689897225] 09:19:59,112 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower24] 09:19:59,112 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - follower24: terminating thread VirtualThread[#337,follower24-writer-9]/waiting 09:19:59,113 AM [follower24-writer-9] [DEBUG] JournalWriteTask - follower24: received 1 commands after 5.514 s 09:19:59,113 AM [follower24-writer-9] [DEBUG] JournalWriteTask - follower24: completed 1 commands in 37.89 μs 09:19:59,113 AM [follower24-writer-9] [DEBUG] JournalWriteTask - follower24: journal writer stopped 09:19:59,113 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower24: journal closed 09:19:59,113 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower24: journal closed 09:19:59,113 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower24: stopped executor with 0 remaining tasks 09:19:59,113 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower24: stopped executor with 0 remaining tasks 09:19:59,114 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader23-collector27#1707710397] 09:19:59,114 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader23] 09:19:59,114 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - leader23: terminating thread VirtualThread[#340,leader23-writer-10]/waiting 09:19:59,115 AM [leader23-writer-10] [DEBUG] JournalWriteTask - leader23: received 1 commands after 5.508 s 09:19:59,115 AM [leader23-writer-10] [DEBUG] JournalWriteTask - leader23: completed 1 commands in 36.62 μs 09:19:59,115 AM [leader23-writer-10] [DEBUG] JournalWriteTask - leader23: journal writer stopped 09:19:59,115 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader23: journal closed 09:19:59,115 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader23: journal closed 09:19:59,115 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: stopped executor with 0 remaining tasks 09:19:59,115 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader23: stopped executor with 0 remaining tasks 09:19:59,121 AM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithEmptyLeaderLogAfterNonPersistentLeaderRestart starting [WARN] [07/05/2025 09:19:59.123] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower29]: org.opendaylight.controller.cluster.common.actor.Monitor@cd0d0bd 09:19:59,123 AM [test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower29] to Actor[pekko://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][07/05/2025 09:19:59.123] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower29] to Actor[pekko://test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:59,125 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - follower29: Starting recovery with journal batch size 1000 09:19:59,125 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower29: started executor 09:19:59,125 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower29: started executor 09:19:59,126 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower29: metadata journal open: replayFrom=1 applyTo=0 09:19:59,127 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower29: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:19:59,127 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower29: journal open: applyTo=0 09:19:59,127 AM [follower29-writer-11] [DEBUG] JournalWriteTask - follower29: journal writer started 09:19:59,127 AM [follower29-writer-11] [DEBUG] JournalWriteTask - follower29: waiting to receive commands [WARN] [07/05/2025 09:19:59.129] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader28]: org.opendaylight.controller.cluster.common.actor.Monitor@64f288bc [INFO] [pekkoDeadLetter][07/05/2025 09:19:59.129] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader28] to Actor[pekko://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:59,129 AM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader28] to Actor[pekko://test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:19:59,130 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActor - leader28: Starting recovery with journal batch size 1000 09:19:59,130 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader28: started executor 09:19:59,131 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader28: started executor 09:19:59,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/TermInfo.properties does not exist 09:19:59,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/journal-v1-1.log 09:19:59,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/metadata-v1-1.log 09:19:59,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: no eligible files found 09:19:59,131 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower29: Set currentTerm=1, votedFor=leader28 09:19:59,131 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader28: metadata journal open: replayFrom=1 applyTo=0 09:19:59,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - follower29: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:19:59,132 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PeerInfos - follower29: Updated server config: isVoting: false, peers: [PeerInfo [id=leader28, address=pekko://test/user/leader28, votingState=VOTING]] 09:19:59,132 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower29: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:19:59,132 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] EntryJournalV1 - leader28: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:19:59,132 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRaftStorage - leader28: journal open: applyTo=0 09:19:59,132 AM [leader28-writer-12] [DEBUG] JournalWriteTask - leader28: journal writer started 09:19:59,132 AM [leader28-writer-12] [DEBUG] JournalWriteTask - leader28: waiting to receive commands 09:19:59,134 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/TermInfo.properties does not exist 09:19:59,134 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/journal-v1-1.log 09:19:59,134 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/metadata-v1-1.log 09:19:59,134 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: no eligible files found 09:19:59,134 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=1, votedFor=leader28 09:19:59,134 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRecovery - leader28: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:19:59,135 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PeerInfos - leader28: Updated server config: isVoting: true, peers: [PeerInfo [id=follower29, address=pekko://test/user/follower29, votingState=NON_VOTING]] 09:19:59,135 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader28: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:19:59,135 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower29: Set currentTerm=1, votedFor=leader28 09:19:59,135 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower29: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader28} 09:19:59,135 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower29: taking snapshot to clear Pekko persistence to 2 09:19:59,135 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/snapshot-000000006868EE3F-081612C8.tmp 09:19:59,139 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=1, votedFor=leader28 09:19:59,139 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader28: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader28} 09:19:59,139 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader28: taking snapshot to clear Pekko persistence to 2 09:19:59,139 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/snapshot-000000006868EE3F-08565EBB.tmp 09:19:59,140 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/snapshot-000000006868EE3F-081612C8.v1 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/snapshot-000000006868EE3F-081612C8.v1 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/TermInfo.properties 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/journal-v1-1.log 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/metadata-v1-1.log 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower29: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:19:59.135664328Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/follower29/snapshot-000000006868EE3F-081612C8.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower29: Pekko recovery completed and can restore from snapshot 09:19:59,141 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower29: no journal: skipping journal recovery 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/snapshot-000000006868EE3F-08565EBB.v1 09:19:59,142 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower29: ignoring DeleteMessagesSuccess(2) 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/snapshot-000000006868EE3F-08565EBB.v1 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/TermInfo.properties 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/journal-v1-1.log 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/metadata-v1-1.log 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:19:59.139878075Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest11251426055315649220/leader28/snapshot-000000006868EE3F-08565EBB.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader28: Pekko recovery completed and can restore from snapshot 09:19:59,142 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader28: no journal: skipping journal recovery 09:19:59,143 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader28: ignoring DeleteMessagesSuccess(2) 09:19:59,624 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:19:59,624 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader28 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.247:17355, Removed) 09:19:59,624 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader28 (Follower): Received ElectionTimeout - switching to Candidate 09:19:59,625 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader28 (Candidate): Election: Candidate has following voting peers: [] 09:19:59,629 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader28: Set currentTerm=2, votedFor=leader28 09:19:59,630 AM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader28 (Candidate): Starting new election term 2 09:19:59,630 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader28 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:19:59,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader28 (Candidate): Received ElectionTimeout 09:19:59,630 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader28 (Leader): Election: Leader has following peers: [follower29] 09:19:59,631 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader28 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:20:04,676 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower29-collector31#-1361929338] 09:20:04,687 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower29] 09:20:04,688 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - follower29: terminating thread VirtualThread[#341,follower29-writer-11]/waiting 09:20:04,688 AM [follower29-writer-11] [DEBUG] JournalWriteTask - follower29: received 1 commands after 5.561 s 09:20:04,688 AM [follower29-writer-11] [DEBUG] JournalWriteTask - follower29: completed 1 commands in 36.95 μs 09:20:04,688 AM [follower29-writer-11] [DEBUG] JournalWriteTask - follower29: journal writer stopped 09:20:04,688 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower29: journal closed 09:20:04,688 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower29: journal closed 09:20:04,689 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower29: stopped executor with 0 remaining tasks 09:20:04,689 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower29: stopped executor with 0 remaining tasks 09:20:04,689 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader28-collector32#-1267433459] 09:20:04,689 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader28] 09:20:04,690 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - leader28: terminating thread VirtualThread[#342,leader28-writer-12]/waiting 09:20:04,690 AM [leader28-writer-12] [DEBUG] JournalWriteTask - leader28: received 1 commands after 5.557 s 09:20:04,690 AM [leader28-writer-12] [DEBUG] JournalWriteTask - leader28: completed 1 commands in 20.07 μs 09:20:04,690 AM [leader28-writer-12] [DEBUG] JournalWriteTask - leader28: journal writer stopped 09:20:04,690 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader28: journal closed 09:20:04,690 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader28: journal closed 09:20:04,690 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: stopped executor with 0 remaining tasks 09:20:04,690 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader28: stopped executor with 0 remaining tasks 09:20:04,695 AM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithOneMoreLeaderLogEntryAfterNonPersistentLeaderRestart starting [WARN] [07/05/2025 09:20:04.697] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower34]: org.opendaylight.controller.cluster.common.actor.Monitor@689b46e7 09:20:04,697 AM [test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower34] to Actor[pekko://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][07/05/2025 09:20:04.697] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower34] to Actor[pekko://test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:20:04,698 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - follower34: Starting recovery with journal batch size 1000 09:20:04,698 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower34: started executor 09:20:04,698 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower34: started executor 09:20:04,699 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower34: metadata journal open: replayFrom=1 applyTo=0 09:20:04,700 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower34: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:20:04,700 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower34: journal open: applyTo=0 09:20:04,700 AM [follower34-writer-13] [DEBUG] JournalWriteTask - follower34: journal writer started 09:20:04,700 AM [follower34-writer-13] [DEBUG] JournalWriteTask - follower34: waiting to receive commands 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/TermInfo.properties does not exist 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/journal-v1-1.log 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/metadata-v1-1.log 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: no eligible files found 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower34: Set currentTerm=1, votedFor=leader33 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - follower34: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:20:04,702 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PeerInfos - follower34: Updated server config: isVoting: false, peers: [PeerInfo [id=leader33, address=pekko://test/user/leader33, votingState=VOTING]] 09:20:04,703 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower34: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 [WARN] [07/05/2025 09:20:04.704] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader33]: org.opendaylight.controller.cluster.common.actor.Monitor@3def1660 [INFO] [pekkoDeadLetter][07/05/2025 09:20:04.704] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader33] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:20:04,704 AM [test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader33] to Actor[pekko://test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:20:04,705 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - leader33: Starting recovery with journal batch size 1000 09:20:04,705 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader33: started executor 09:20:04,705 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader33: started executor 09:20:04,706 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower34: Set currentTerm=1, votedFor=leader33 09:20:04,706 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower34: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader33} 09:20:04,706 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower34: taking snapshot to clear Pekko persistence to 2 09:20:04,706 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/snapshot-000000006868EE44-2A171E32.tmp 09:20:04,706 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader33: metadata journal open: replayFrom=1 applyTo=0 09:20:04,707 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader33: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:20:04,707 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader33: journal open: applyTo=0 09:20:04,707 AM [leader33-writer-14] [DEBUG] JournalWriteTask - leader33: journal writer started 09:20:04,707 AM [leader33-writer-14] [DEBUG] JournalWriteTask - leader33: waiting to receive commands 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/snapshot-000000006868EE44-2A171E32.v1 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/snapshot-000000006868EE44-2A171E32.v1 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/TermInfo.properties 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/journal-v1-1.log 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/metadata-v1-1.log 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower34: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:20:04.706158130Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/follower34/snapshot-000000006868EE44-2A171E32.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower34: Pekko recovery completed and can restore from snapshot 09:20:04,708 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower34: no journal: skipping journal recovery 09:20:04,709 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower34: ignoring DeleteMessagesSuccess(2) 09:20:04,709 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/TermInfo.properties does not exist 09:20:04,709 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/journal-v1-1.log 09:20:04,710 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/metadata-v1-1.log 09:20:04,710 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: no eligible files found 09:20:04,710 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader33: Set currentTerm=1, votedFor=leader33 09:20:04,710 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader33: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:20:04,710 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PeerInfos - leader33: Updated server config: isVoting: true, peers: [PeerInfo [id=follower34, address=pekko://test/user/follower34, votingState=NON_VOTING]] 09:20:04,710 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - leader33: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:20:04,712 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader33: Set currentTerm=1, votedFor=leader33 09:20:04,712 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - leader33: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader33} 09:20:04,712 AM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - leader33: taking snapshot to clear Pekko persistence to 2 09:20:04,712 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/snapshot-000000006868EE44-2A78D0E4.tmp 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/snapshot-000000006868EE44-2A78D0E4.v1 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/snapshot-000000006868EE44-2A78D0E4.v1 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/TermInfo.properties 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/journal-v1-1.log 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/metadata-v1-1.log 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader33: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:20:04.712560868Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest2707789208062011449/leader33/snapshot-000000006868EE44-2A78D0E4.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader33: Pekko recovery completed and can restore from snapshot 09:20:04,714 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader33: no journal: skipping journal recovery 09:20:04,715 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader33: ignoring DeleteMessagesSuccess(2) 09:20:05,145 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:20:05,145 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader33 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.247:17355, Removed) 09:20:05,145 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader33 (Follower): Received ElectionTimeout - switching to Candidate 09:20:05,145 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader33 (Candidate): Election: Candidate has following voting peers: [] 09:20:05,150 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader33: Set currentTerm=2, votedFor=leader33 09:20:05,150 AM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader33 (Candidate): Starting new election term 2 09:20:05,150 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader33 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:20:05,150 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader33 (Candidate): Received ElectionTimeout 09:20:05,150 AM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader33 (Leader): Election: Leader has following peers: [follower34] 09:20:05,150 AM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader33 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:20:10,199 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower34-collector36#1456291322] 09:20:10,211 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower34] 09:20:10,211 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - follower34: terminating thread VirtualThread[#343,follower34-writer-13]/waiting 09:20:10,212 AM [follower34-writer-13] [DEBUG] JournalWriteTask - follower34: received 1 commands after 5.512 s 09:20:10,212 AM [follower34-writer-13] [DEBUG] JournalWriteTask - follower34: completed 1 commands in 37.34 μs 09:20:10,212 AM [follower34-writer-13] [DEBUG] JournalWriteTask - follower34: journal writer stopped 09:20:10,213 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower34: journal closed 09:20:10,213 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower34: journal closed 09:20:10,213 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower34: stopped executor with 0 remaining tasks 09:20:10,213 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower34: stopped executor with 0 remaining tasks 09:20:10,213 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader33-collector37#20262870] 09:20:10,214 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader33] 09:20:10,214 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] PekkoRaftStorage - leader33: terminating thread VirtualThread[#344,leader33-writer-14]/waiting 09:20:10,215 AM [leader33-writer-14] [DEBUG] JournalWriteTask - leader33: received 1 commands after 5.508 s 09:20:10,215 AM [leader33-writer-14] [DEBUG] JournalWriteTask - leader33: completed 1 commands in 23.41 μs 09:20:10,215 AM [leader33-writer-14] [DEBUG] JournalWriteTask - leader33: journal writer stopped 09:20:10,215 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader33: journal closed 09:20:10,215 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader33: journal closed 09:20:10,215 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader33: stopped executor with 0 remaining tasks 09:20:10,215 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader33: stopped executor with 0 remaining tasks 09:20:10,222 AM [main] [INFO] NonVotingFollowerIntegrationTest - testFollowerResyncWithMoreLeaderLogEntriesAndDownPeerAfterNonPersistentLeaderRestart starting [WARN] [07/05/2025 09:20:10.225] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower39]: org.opendaylight.controller.cluster.common.actor.Monitor@232ac05e 09:20:10,225 AM [test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower39] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. [INFO] [pekkoDeadLetter][07/05/2025 09:20:10.225] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower39] to Actor[pekko://test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:20:10,226 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - follower39: Starting recovery with journal batch size 1000 09:20:10,227 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower39: started executor 09:20:10,227 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower39: started executor 09:20:10,228 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower39: metadata journal open: replayFrom=1 applyTo=0 09:20:10,229 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - follower39: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:20:10,229 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - follower39: journal open: applyTo=0 09:20:10,229 AM [follower39-writer-15] [DEBUG] JournalWriteTask - follower39: journal writer started 09:20:10,229 AM [follower39-writer-15] [DEBUG] JournalWriteTask - follower39: waiting to receive commands [WARN] [07/05/2025 09:20:10.231] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader38]: org.opendaylight.controller.cluster.common.actor.Monitor@58832089 09:20:10,231 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/TermInfo.properties does not exist 09:20:10,231 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/journal-v1-1.log 09:20:10,231 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/metadata-v1-1.log 09:20:10,231 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: no eligible files found 09:20:10,231 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower39: Set currentTerm=1, votedFor=leader38 09:20:10,232 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRecovery - follower39: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:20:10,232 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PeerInfos - follower39: Updated server config: isVoting: false, peers: [PeerInfo [id=leader38, address=pekko://test/user/leader38, votingState=VOTING]] 09:20:10,232 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower39: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:20:10,233 AM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - leader38: Starting recovery with journal batch size 1000 09:20:10,233 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader38: started executor 09:20:10,233 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader38: started executor 09:20:10,234 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader38: metadata journal open: replayFrom=1 applyTo=0 09:20:10,234 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - follower39: Set currentTerm=1, votedFor=leader38 [INFO] [pekkoDeadLetter][07/05/2025 09:20:10.234] [test-pekko.actor.default-dispatcher-11] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader38] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:20:10,234 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower39: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader38} 09:20:10,234 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - follower39: taking snapshot to clear Pekko persistence to 2 09:20:10,234 AM [test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader38] to Actor[pekko://test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:20:10,234 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/snapshot-000000006868EE4A-0E01190A.tmp 09:20:10,235 AM [test-pekko.actor.default-dispatcher-14] [DEBUG] EntryJournalV1 - leader38: journal open: firstIndex=1 lastIndex=0 replayFrom=1 fileEntries=0 09:20:10,235 AM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRaftStorage - leader38: journal open: applyTo=0 09:20:10,235 AM [leader38-writer-16] [DEBUG] JournalWriteTask - leader38: journal writer started 09:20:10,235 AM [leader38-writer-16] [DEBUG] JournalWriteTask - leader38: waiting to receive commands 09:20:10,237 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/TermInfo.properties does not exist 09:20:10,237 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/journal-v1-1.log 09:20:10,237 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/metadata-v1-1.log 09:20:10,237 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: no eligible files found 09:20:10,237 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader38: Set currentTerm=1, votedFor=leader38 09:20:10,237 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PekkoRecovery - leader38: Received ReplicatedLogEntry for recovery: index: 0, size: 141 09:20:10,238 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PeerInfos - leader38: Updated server config: isVoting: true, peers: [PeerInfo [id=follower39, address=pekko://test/user/follower39, votingState=NON_VOTING]] 09:20:10,238 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader38: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:20:10,238 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/snapshot-000000006868EE4A-0E01190A.v1 09:20:10,238 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/TermInfo.properties 09:20:10,239 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/journal-v1-1.log 09:20:10,239 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/metadata-v1-1.log 09:20:10,239 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/snapshot-000000006868EE4A-0E01190A.v1 09:20:10,239 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:20:10.234952970Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/follower39/snapshot-000000006868EE4A-0E01190A.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:20:10,239 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower39: Pekko recovery completed and can restore from snapshot 09:20:10,239 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower39: no journal: skipping journal recovery 09:20:10,240 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - follower39: ignoring DeleteMessagesSuccess(2) 09:20:10,240 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader38: Set currentTerm=1, votedFor=leader38 09:20:10,240 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader38: Local TermInfo store seeded with TermInfo{term=1, votedFor=leader38} 09:20:10,240 AM [test-pekko.actor.default-dispatcher-11] [INFO] PekkoRecovery - leader38: taking snapshot to clear Pekko persistence to 2 09:20:10,240 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/snapshot-000000006868EE4A-0E57290C.tmp 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/snapshot-000000006868EE4A-0E57290C.v1 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/TermInfo.properties 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/journal-v1-1.log 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/snapshot-000000006868EE4A-0E57290C.v1 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/metadata-v1-1.log 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftStorage - leader38: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=-1, term=-1], timestamp=2025-07-05T09:20:10.240593164Z, file=/tmp/org.opendaylight.controller.cluster.raft.NonVotingFollowerIntegrationTest8663443802817533139/leader38/snapshot-000000006868EE4A-0E57290C.v1, size=88, entryCompress=NONE, stateCompress=NONE} 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader38: Pekko recovery completed and can restore from snapshot 09:20:10,242 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader38: no journal: skipping journal recovery 09:20:10,243 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] RaftActor - leader38: ignoring DeleteMessagesSuccess(2) 09:20:10,684 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy 09:20:10,684 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader38 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.247:17355, Removed) 09:20:10,684 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Follower - leader38 (Follower): Received ElectionTimeout - switching to Candidate 09:20:10,685 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader38 (Candidate): Election: Candidate has following voting peers: [] 09:20:10,689 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] PropertiesTermInfoStore - leader38: Set currentTerm=2, votedFor=leader38 09:20:10,689 AM [test-pekko.actor.default-dispatcher-11] [INFO] Candidate - leader38 (Candidate): Starting new election term 2 09:20:10,689 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader38 (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:20:10,690 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] Candidate - leader38 (Candidate): Received ElectionTimeout 09:20:10,690 AM [test-pekko.actor.default-dispatcher-11] [DEBUG] AbstractLeader - leader38 (Leader): Election: Leader has following peers: [follower39] 09:20:10,690 AM [test-pekko.actor.default-dispatcher-11] [INFO] RaftActorBehavior - leader38 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:20:15,735 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower39-collector41#-103008764] 09:20:15,747 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower39] 09:20:15,747 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - follower39: terminating thread VirtualThread[#345,follower39-writer-15]/waiting 09:20:15,747 AM [follower39-writer-15] [DEBUG] JournalWriteTask - follower39: received 1 commands after 5.518 s 09:20:15,748 AM [follower39-writer-15] [DEBUG] JournalWriteTask - follower39: completed 1 commands in 49.32 μs 09:20:15,748 AM [follower39-writer-15] [DEBUG] JournalWriteTask - follower39: journal writer stopped 09:20:15,748 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - follower39: journal closed 09:20:15,748 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - follower39: journal closed 09:20:15,748 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: stopped executor with 0 remaining tasks 09:20:15,748 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - follower39: stopped executor with 0 remaining tasks 09:20:15,749 AM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader38-collector42#-600641875] 09:20:15,749 AM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader38] 09:20:15,749 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] PekkoRaftStorage - leader38: terminating thread VirtualThread[#346,leader38-writer-16]/waiting 09:20:15,749 AM [leader38-writer-16] [DEBUG] JournalWriteTask - leader38: received 1 commands after 5.514 s 09:20:15,750 AM [leader38-writer-16] [DEBUG] JournalWriteTask - leader38: completed 1 commands in 20.61 μs 09:20:15,750 AM [leader38-writer-16] [DEBUG] JournalWriteTask - leader38: journal writer stopped 09:20:15,750 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] EntryJournalV1 - leader38: journal closed 09:20:15,750 AM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRaftStorage - leader38: journal closed 09:20:15,750 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader38: stopped executor with 0 remaining tasks 09:20:15,750 AM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader38: stopped executor with 0 remaining tasks 09:20:15,753 AM [test-pekko.actor.default-dispatcher-11] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [INFO] [07/05/2025 09:20:15.753] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [WARN] [07/05/2025 09:20:15.755] [test-pekko.actor.default-dispatcher-5] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.171.247:17355) [INFO] [07/05/2025 09:20:15.756] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - Exiting completed [INFO] [07/05/2025 09:20:15.756] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - Shutting down... 09:20:15,756 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - Exiting completed 09:20:15,756 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - Shutting down... 09:20:15,756 AM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.247:17355] - Successfully shut down [INFO] [07/05/2025 09:20:15.756] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.247:17355] - Successfully shut down 09:20:15,759 AM [test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. [INFO] [07/05/2025 09:20:15.759] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.171.247:17355/system/remoting-terminator] Shutting down remote daemon. 09:20:15,759 AM [test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [INFO] [07/05/2025 09:20:15.759] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.171.247:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports. [INFO] [07/05/2025 09:20:15.761] [test-pekko.remote.default-remote-dispatcher-7] [pekko://test@10.30.171.247:17355/system/remoting-terminator] Remoting shut down. 09:20:15,762 AM [test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [DEBUG] [07/05/2025 09:20:15.762] [test-pekko.actor.internal-dispatcher-2] [EventStream] shutting down: StandardOutLogger