11:35:21,499 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:22,47 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [4046474958504811861] 11:35:22,71 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:35:22,280 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:35:22,280 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:35:22,340 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:22,341 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:22,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:35:22,704 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:35:22,724 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:35:22,736 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:22,749 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:23,628 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 11:35:23,756 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:35:23,756 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:35:23,763 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:23,818 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:35:23,825 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 11:35:23,835 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 11:35:23,880 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 11:35:23,907 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:23,907 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 11:35:23,908 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 11:35:23,921 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@16089d20} 11:35:23,959 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 11:35:23,979 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:23,979 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 11:35:23,983 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 11:35:23,983 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 11:35:23,984 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-41219929] 11:35:23,984 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 11:35:23,985 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 11:35:23,986 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 11:35:23,986 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#1840926497] 11:35:23,988 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#1840926497] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:23,990 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-41219929] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,22 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Shard created, persistent : true 11:35:24,24 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Shard created, persistent : true 11:35:24,28 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,29 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,29 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,80 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,132 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,133 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,133 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,145 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1605257103] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,147 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1605257103 created and ready for shard:member-1-shard-people-config_testBackupDatastore 11:35:24,148 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-config_testBackupDatastore: Starting recovery with journal batch size 1 11:35:24,155 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-795196301 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 11:35:24,156 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testBackupDatastore: Starting recovery with journal batch size 1 11:35:24,156 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-795196301] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,184 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,184 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,185 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,235 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,236 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,236 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,286 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,287 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,287 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,314 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal open: applyTo=0 11:35:24,314 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal open: applyTo=0 11:35:24,337 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,338 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,338 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 11:35:24,339 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:24,339 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:24,351 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:24,351 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:24,358 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 11:35:24,363 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 11:35:24,368 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:35:24,369 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 11:35:24,369 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:35:24,369 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 11:35:24,373 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,374 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,375 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 11:35:24,375 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,375 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,375 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 11:35:24,389 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,390 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,390 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-41219929] 11:35:24,395 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 11:35:24,395 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:24,395 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,395 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 11:35:24,398 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:24,398 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 11:35:24,398 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 11:35:24,398 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:24,398 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,398 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,399 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 11:35:24,399 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:24,399 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 11:35:24,401 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 11:35:24,401 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 11:35:24,402 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 11:35:24,402 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 11:35:24,409 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 11:35:24,410 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@522f92be 11:35:24,410 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 11:35:24,410 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 11:35:24,410 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c432f6d 11:35:24,410 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 11:35:24,410 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 11:35:24,461 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,462 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:35:24,463 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#1840926497] 11:35:24,464 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:24,474 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:24,486 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 11:35:24,489 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1504386249] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,489 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:35:24,490 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:35:24,495 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:35:24,495 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 11:35:24,496 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 11:35:24,496 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:24,496 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 11:35:24,496 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 11:35:24,499 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 11:35:24,501 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@68966a3f} 11:35:24,501 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 11:35:24,501 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:24,502 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 11:35:24,502 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 11:35:24,503 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 11:35:24,503 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-2041619061] 11:35:24,503 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 11:35:24,503 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Shard created, persistent : true 11:35:24,503 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 11:35:24,503 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 11:35:24,503 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1112104695] 11:35:24,504 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Shard created, persistent : true 11:35:24,504 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-oper_testBackupDatastore: Starting recovery with journal batch size 1 11:35:24,505 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-oper_testBackupDatastore: Starting recovery with journal batch size 1 11:35:24,506 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal open: applyTo=0 11:35:24,507 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1336796394 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 11:35:24,507 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-2041619061] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,507 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-100112271 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 11:35:24,507 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal open: applyTo=0 11:35:24,508 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1112104695] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,508 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1336796394] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,508 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:24,509 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:24,510 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:24,511 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:24,511 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-100112271] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:24,512 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 11:35:24,513 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:35:24,513 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,513 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 11:35:24,513 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,513 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 11:35:24,514 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 11:35:24,514 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:35:24,514 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,514 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 11:35:24,515 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,515 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 11:35:24,517 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 11:35:24,517 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:24,517 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,518 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,518 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:24,518 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 11:35:24,518 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 11:35:24,519 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 11:35:24,519 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:24,519 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 11:35:24,519 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:24,519 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 11:35:24,519 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 11:35:24,519 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:24,519 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 11:35:24,519 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 11:35:24,520 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 11:35:24,520 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 11:35:24,520 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 11:35:24,520 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 11:35:24,520 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7991b0a5 11:35:24,520 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 11:35:24,521 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3245cb48 11:35:24,521 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 11:35:24,521 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 11:35:24,557 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1504386249] 11:35:24,557 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:35:24,557 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-2041619061] 11:35:24,608 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1504386249] 11:35:24,609 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:35:24,609 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1112104695] 11:35:24,631 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 11:35:24,632 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 11:35:24,634 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 11:35:24,642 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 11:35:24,644 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 11:35:24,658 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@773d426f, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 11:35:24,657 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6c1170f, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 11:35:24,673 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5f198358, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 11:35:24,673 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 11:35:24,673 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@25c5b804, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 11:35:24,675 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 11:35:24,730 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 11:35:24,763 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#574295132] 11:35:24,765 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:24,765 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-41219929] 11:35:24,798 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Stopping Shard 11:35:24,802 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal closed 11:35:24,808 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 11:35:24,809 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 11:35:24,810 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 11:35:24,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@33076f42, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 11:35:24,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7dac5b06, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 11:35:24,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 11:35:24,812 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 11:35:24,812 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 11:35:24,814 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@11e2cd70, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 11:35:25,32 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 11:35:25,35 AM [cluster-test-pekko.actor.default-dispatcher-11] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 11:35:25,45 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 11:35:25,49 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 11:35:25,52 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 11:35:25,55 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6236bb27 11:35:25,55 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6236bb27 11:35:25,56 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:35:25,56 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Stopping Shard 11:35:25,57 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal closed 11:35:25,61 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:35:25,62 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:35:25,66 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 11:35:25,67 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 11:35:25,67 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4a03e050 11:35:25,67 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4a03e050 11:35:25,68 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 11:35:25,68 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Stopping Shard 11:35:25,68 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@37f76014 11:35:25,68 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@37f76014 11:35:25,69 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal closed 11:35:25,69 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Stopping Shard 11:35:25,69 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:35:25,70 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal closed 11:35:25,70 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:35:25,70 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:35:25,79 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:25,80 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:25,108 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:35:25,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:25,197 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-285212533914412641] 11:35:25,199 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:35:25,203 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:35:25,203 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:35:25,205 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:25,205 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:25,232 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,235 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:35:25,235 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:35:25,235 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:35:25,238 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,243 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 11:35:25,256 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:35:25,257 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:35:25,257 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-549349197] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,263 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:35:25,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 11:35:25,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 11:35:25,264 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:25,264 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 11:35:25,264 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 11:35:25,268 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 11:35:25,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@6a36dd43} 11:35:25,269 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 11:35:25,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:25,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 11:35:25,270 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 11:35:25,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 11:35:25,271 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-571568366] 11:35:25,271 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 11:35:25,272 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 11:35:25,272 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:25,274 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 11:35:25,274 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-571568366] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,275 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-49364989 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 11:35:25,275 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-49364989] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,275 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 11:35:25,276 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#581690478] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,277 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#1146112996 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 11:35:25,277 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#1146112996] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,277 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:25,278 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 11:35:25,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:25,280 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:25,281 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 11:35:25,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 11:35:25,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#581690478] 11:35:25,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:25,283 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:25,284 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:25,285 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,285 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,285 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 11:35:25,286 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:35:25,286 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 11:35:25,287 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:25,287 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,287 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:25,288 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,288 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:25,288 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 11:35:25,288 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 11:35:25,289 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 11:35:25,289 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:35:25,289 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 11:35:25,289 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f9aae71 11:35:25,289 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:25,290 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 11:35:25,290 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 11:35:25,291 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 11:35:25,291 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 11:35:25,291 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25cc244e 11:35:25,291 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 11:35:25,291 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:25,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-549349197] 11:35:25,320 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:25,320 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-571568366] 11:35:25,371 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-549349197] 11:35:25,371 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:35:25,372 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#581690478] 11:35:25,372 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,378 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,392 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 11:35:25,396 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#325290223] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,397 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:35:25,397 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:35:25,399 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:35:25,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:25,400 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 11:35:25,400 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 11:35:25,400 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:25,401 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:25,401 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 11:35:25,401 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 11:35:25,401 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@79cc2971} 11:35:25,401 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:25,401 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:25,404 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 11:35:25,404 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#719143769] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,405 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:25,405 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1475302060 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:25,405 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1475302060] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,406 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 11:35:25,406 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:25,406 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:25,407 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:25,408 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:25,408 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#719143769] 11:35:25,408 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 11:35:25,409 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 11:35:25,409 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 11:35:25,409 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1777788904] 11:35:25,410 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 11:35:25,410 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1777788904] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,411 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:25,411 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-717750012 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 11:35:25,411 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-717750012] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,411 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:25,412 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,412 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,412 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 11:35:25,412 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 11:35:25,412 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:35:25,412 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 11:35:25,413 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:25,413 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:25,415 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 11:35:25,415 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:25,415 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,415 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,416 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:25,416 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:25,416 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 11:35:25,416 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 11:35:25,416 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 11:35:25,416 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:25,416 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 11:35:25,416 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42e71931 11:35:25,416 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 11:35:25,416 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:35:25,417 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 11:35:25,417 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,417 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,417 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 11:35:25,420 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 11:35:25,420 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:25,420 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,420 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,420 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 11:35:25,420 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:25,420 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:25,421 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 11:35:25,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 11:35:25,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 11:35:25,421 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11f08993 11:35:25,421 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 11:35:25,421 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 11:35:25,451 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#325290223] 11:35:25,452 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:35:25,452 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#719143769] 11:35:25,503 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#325290223] 11:35:25,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:35:25,503 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1777788904] 11:35:25,509 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@70283a49] 11:35:25,515 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 11:35:25,517 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 11:35:25,518 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:25,518 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#1386213079 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 11:35:25,520 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 11:35:25,522 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:25,522 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:25,526 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:25,526 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 11:35:25,526 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,526 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 11:35:25,527 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,527 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 11:35:25,530 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 11:35:25,531 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:25,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:25,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:25,531 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 11:35:25,531 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:25,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:25,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 11:35:25,531 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 11:35:25,532 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 11:35:25,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f286dc2 11:35:25,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 11:35:25,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:25,571 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-549349197] 11:35:25,572 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 11:35:25,572 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#956784241] 11:35:25,592 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:25,623 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [3033083213175635949] 11:35:25,624 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:35:25,630 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:25,631 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:35:25,633 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:25,633 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:25,686 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,689 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,702 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 11:35:25,716 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:35:25,716 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:35:25,719 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-994094337] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,727 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:35:25,730 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 11:35:25,735 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5406644d} 11:35:25,735 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 11:35:25,736 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:25,738 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:25,741 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 11:35:25,748 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#235476276] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:25,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:35:25,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:35:25,749 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:35:25,754 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 11:35:25,755 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5ac180bd} 11:35:25,755 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 11:35:25,755 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:26,36 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:35:26,88 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:26,95 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:26,95 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 11:35:26,95 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:26,95 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:26,229 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:35:26,229 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:26,230 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 11:35:26,230 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:26,230 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 11:35:26,230 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 11:35:26,230 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 11:35:26,230 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 11:35:26,231 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:26,269 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3040e2d4] 11:35:26,271 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 11:35:26,273 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 11:35:26,274 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1947095223] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,276 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#770304385 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 11:35:26,276 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#770304385] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,275 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-2-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:26,277 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@485b1bf6] 11:35:26,277 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 11:35:26,277 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 11:35:26,279 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:26,279 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:26,280 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#535578070] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,281 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Shard created, persistent : true 11:35:26,282 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1886257024 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 11:35:26,282 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1886257024] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,283 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:26,284 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:26,284 AM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 11:35:26,285 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 11:35:26,285 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 11:35:26,285 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 11:35:26,285 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 11:35:26,286 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 11:35:26,288 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:26,288 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,288 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 11:35:26,288 AM [cluster-test-shard-dispatcher-10] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:35:26,289 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 11:35:26,289 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 11:35:26,289 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 11:35:26,289 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 11:35:26,290 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:26,299 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 11:35:26,300 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 11:35:26,300 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:35:26,300 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 11:35:26,300 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 11:35:26,300 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 11:35:26,289 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:26,307 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 11:35:26,307 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 11:35:26,308 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 11:35:26,308 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:35:26,309 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 11:35:26,309 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 11:35:26,309 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 11:35:26,310 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal open: applyTo=0 11:35:26,313 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:26,314 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:26,320 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:35:26,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-571568366, primaryVersion=13] 11:35:26,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 11:35:26,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#581690478, primaryVersion=13] 11:35:26,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 11:35:26,322 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#956784241, primaryVersion=13] 11:35:26,322 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:35:26,323 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:26,323 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#719143769, primaryVersion=13] 11:35:26,323 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 11:35:26,323 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1777788904, primaryVersion=13] 11:35:26,323 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 11:35:26,323 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:26,323 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 11:35:26,323 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,323 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 11:35:26,324 AM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:35:26,324 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 11:35:26,324 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 11:35:26,324 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 11:35:26,324 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 11:35:26,334 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-571568366 for shard member-2-shard-cars-config_testAddReplicasForAllShards 11:35:26,335 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#215246900] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,334 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#719143769 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 11:35:26,335 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 11:35:26,336 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1583152141] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,336 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 11:35:26,336 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#772066502 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 11:35:26,336 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:26,336 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#772066502] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,337 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-1119714513 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 11:35:26,337 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-1119714513] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:26,338 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 11:35:26,339 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:26,340 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 11:35:26,340 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 11:35:26,341 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:26,341 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 11:35:26,341 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#-922537363 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 11:35:26,342 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#-1222774090 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 11:35:26,342 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 11:35:26,342 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 11:35:26,343 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1034) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:229) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 11:35:26,344 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=no-leader}, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1034) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:229) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 11:35:26,348 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:26,348 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:26,351 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:26,351 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1777788904 for shard member-2-shard-people-oper_testAddReplicasForAllShards 11:35:26,352 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:26,352 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,352 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:26,352 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:26,352 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 11:35:26,354 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 11:35:26,357 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:26,357 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:26,355 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 11:35:26,355 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 11:35:26,358 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 11:35:26,362 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:26,363 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:35:26,363 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 11:35:26,363 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:26,363 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,363 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 11:35:26,363 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 11:35:26,363 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#581690478 for shard member-2-shard-people-config_testAddReplicasForAllShards 11:35:26,364 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 11:35:26,364 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 11:35:26,364 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#956784241 for shard member-2-shard-pets-config_testAddReplicasForAllShards 11:35:26,364 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:35:26,364 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 11:35:26,365 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 11:35:26,367 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 11:35:26,368 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 11:35:26,368 AM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 11:35:26,368 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:26,368 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:26,369 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:26,369 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:35:26,369 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 11:35:26,369 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:26,369 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,369 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 11:35:26,369 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 11:35:26,371 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 11:35:26,371 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:35:26,371 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:35:26,371 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,371 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 11:35:26,371 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:35:26,371 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 11:35:26,372 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 11:35:26,372 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 11:35:26,387 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:26,385 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:26,385 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:26,388 AM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:26,388 AM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:26,388 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:26,388 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:26,391 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:26,392 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:26,392 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:26,411 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:26,411 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:26,413 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:26,413 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:26,406 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:26,417 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.388602198Z 11:35:26,417 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.388579447Z 11:35:26,419 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-oper_testAddReplicasForAllShards 11:35:26,419 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:26,419 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddReplicasForAllShards 11:35:26,419 AM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.388578787Z 11:35:26,420 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:26,420 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:26,420 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@36199821 11:35:26,420 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d58367 11:35:26,420 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 11:35:26,420 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 11:35:26,422 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:26,422 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63a43450 11:35:26,422 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 11:35:26,423 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-pets-config_testAddReplicasForAllShards 11:35:26,423 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7e6a5520 11:35:26,423 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 11:35:26,423 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:26,423 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-config_testAddReplicasForAllShards 11:35:26,425 AM [cluster-test-shard-dispatcher-32] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:26,425 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@653e6d53 11:35:26,425 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:26,425 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 11:35:26,427 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.421712348Z 11:35:26,428 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 11:35:26,428 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:26,430 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.425617940Z 11:35:26,430 AM [cluster-test-shard-dispatcher-31] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 11:35:26,430 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:26,431 AM [cluster-test-shard-dispatcher-32] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.421713088Z 11:35:26,431 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.422447369Z 11:35:26,431 AM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 11:35:26,431 AM [cluster-test-shard-dispatcher-25] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 11:35:26,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:26,431 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:26,432 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.426393292Z 11:35:26,432 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 11:35:26,432 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:26,440 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.388591697Z 11:35:26,441 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-21T11:35:26.392123328Z 11:35:26,456 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#645481082] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 11:35:26,456 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#1353039567] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 11:35:26,456 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-748768126] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 11:35:26,457 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#569922070] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 11:35:26,457 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 24.51 ms 11:35:26,457 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 11:35:26,457 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#876990119] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 11:35:26,457 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 26.73 ms 11:35:26,457 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 28.93 ms 11:35:26,457 AM [cluster-test-shard-dispatcher-31] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 11:35:26,457 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 11:35:26,458 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 26.81 ms 11:35:26,459 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 31.94 ms 11:35:26,461 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#569922070] 11:35:26,463 AM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 11:35:26,465 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 11:35:26,461 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#1353039567] 11:35:26,467 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 11:35:26,461 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-748768126] 11:35:26,461 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#645481082] 11:35:26,467 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 11:35:26,461 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#876990119] 11:35:26,467 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 11:35:26,468 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 11:35:26,471 AM [cluster-test-shard-dispatcher-30] [INFO] AbstractLeader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:26,471 AM [cluster-test-shard-dispatcher-18] [INFO] AbstractLeader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:26,472 AM [cluster-test-shard-dispatcher-19] [INFO] AbstractLeader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:26,473 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:26,478 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 10.50 ms 11:35:26,478 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 10.45 ms 11:35:26,478 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 11.09 ms 11:35:26,478 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 11.64 ms 11:35:26,482 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 17.84 ms 11:35:26,461 AM [cluster-test-shard-dispatcher-25] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 11:35:26,485 AM [cluster-test-shard-dispatcher-19] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:26,497 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#956784241 11:35:26,497 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 11:35:26,498 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 11:35:26,498 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets] 11:35:26,499 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1777788904 11:35:26,499 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 11:35:26,499 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 11:35:26,499 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [people] 11:35:26,503 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 11:35:26,502 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:26,503 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,503 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:26,503 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:26,503 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:26,502 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:26,504 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-571568366 11:35:26,504 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#719143769 11:35:26,504 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 11:35:26,504 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 11:35:26,505 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 11:35:26,505 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars] 11:35:26,505 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 11:35:26,505 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 11:35:26,505 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:35:26,505 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 11:35:26,505 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 11:35:26,505 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 11:35:26,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 11:35:26,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,505 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 11:35:26,506 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,506 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 11:35:26,506 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,507 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:35:26,507 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#581690478 11:35:26,507 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 11:35:26,507 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 11:35:26,508 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 11:35:26,508 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 11:35:26,508 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 11:35:26,508 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:26,508 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77e52f69 11:35:26,508 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@67264783 11:35:26,508 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 11:35:26,508 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 11:35:26,508 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:35:26,508 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 11:35:26,510 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c6510d6 11:35:26,510 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b646bc5 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@375c6fa3 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 11:35:26,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 11:35:26,511 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 11:35:26,511 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 11:35:26,513 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 11:35:26,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 11:35:26,513 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 11:35:26,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 11:35:26,514 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 11:35:26,516 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-994094337] 11:35:26,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:26,518 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#215246900] 11:35:26,521 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#215246900] 11:35:26,522 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-994094337] 11:35:26,522 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:35:26,522 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#946224956] 11:35:26,522 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#946224956] 11:35:26,523 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-994094337] 11:35:26,523 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 11:35:26,523 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1947095223] 11:35:26,523 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1947095223] 11:35:26,523 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#235476276] 11:35:26,523 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:35:26,524 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1583152141] 11:35:26,524 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1583152141] 11:35:26,524 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#235476276] 11:35:26,524 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 11:35:26,524 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-970131570] 11:35:26,524 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-970131570] 11:35:26,525 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 11:35:26,525 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 11:35:26,525 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 11:35:26,525 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 11:35:26,525 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 11:35:26,526 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 11:35:26,526 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Stopping Shard 11:35:26,527 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal closed 11:35:26,527 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal closed 11:35:26,527 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 11:35:26,528 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal closed 11:35:26,528 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:35:26,528 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:35:26,529 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 11:35:26,530 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 11:35:26,530 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 11:35:26,530 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 11:35:26,530 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 11:35:26,532 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Stopping Shard 11:35:26,532 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 11:35:26,532 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 11:35:26,532 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal closed 11:35:26,532 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal closed 11:35:26,532 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal closed 11:35:26,533 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:35:26,533 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:35:26,535 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:26,535 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:26,558 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:26,558 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:26,566 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:35:26,586 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 11:35:26,587 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 11:35:26,587 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 11:35:26,587 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 11:35:26,588 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 11:35:26,592 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@764dafb3 11:35:26,592 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7678d15c 11:35:26,592 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4427c97f 11:35:26,593 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c29cf1b 11:35:26,593 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c29cf1b 11:35:26,593 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 11:35:26,594 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@17dc9a45 11:35:26,594 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@17dc9a45 11:35:26,594 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 11:35:26,594 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12acda4b 11:35:26,594 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12acda4b 11:35:26,594 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 11:35:27,562 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [3033083213175635949] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:35:30,849 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:31,336 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:31,369 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:32,348 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 11:35:32,350 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:32,350 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:33,87 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:33,331 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:33,402 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:33,725 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:34,44 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:34,362 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:34,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:34,683 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:35,2 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:35,324 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:35,443 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:35,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 11:35:35,632 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 11:35:35,632 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 11:35:35,632 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.040 s 11:35:35,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 11:35:35,632 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.041 s 11:35:35,632 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 11:35:35,632 AM [cluster-test-shard-dispatcher-19] [WARN] Leader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,632 AM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.040 s 11:35:35,633 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 11:35:35,633 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,633 AM [cluster-test-shard-dispatcher-30] [WARN] Leader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,633 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:35:35,634 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:35:35,634 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:35,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:35,634 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 11:35:35,634 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:35:35,634 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 11:35:35,634 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:35,634 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 11:35:35,634 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,634 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,635 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 11:35:35,635 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Stopping Shard 11:35:35,635 AM [cluster-test-shard-dispatcher-12] [WARN] Leader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,635 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal closed 11:35:35,635 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal closed 11:35:35,636 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:35:35,636 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:35:35,636 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 11:35:35,636 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 11:35:35,637 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal closed 11:35:35,637 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:35:35,638 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,638 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:35:35,638 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 11:35:35,638 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 11:35:35,639 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 11:35:35,640 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 11:35:35,640 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 11:35:35,640 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:35:35,641 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal closed 11:35:35,642 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 11:35:35,643 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal closed 11:35:35,644 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:35:35,644 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:35:35,645 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:35,646 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:35,646 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:35,648 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:35:35.650] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:35,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:35,713 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4004285426183094035] 11:35:35,713 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:35:35,716 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:35:35,716 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:35:35,718 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:35,718 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:35,739 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:35:35,742 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:35:35,742 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:35:35,744 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,747 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 11:35:35,763 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,771 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:35:35,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:35:35,776 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:35:35,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 11:35:35,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 11:35:35,777 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:35,778 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:35,778 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:35,786 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:35,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@22aa4cd0} 11:35:35,788 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 11:35:35,788 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,789 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:35,789 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 11:35:35,790 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:35,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:35,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:35,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 11:35:35,790 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:35,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:35,791 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-347157027] 11:35:35,791 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,791 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,792 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,793 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,794 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,794 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,795 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-347157027] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#1405565878 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 11:35:35,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#1405565878] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,797 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 11:35:35,797 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-140792676 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 11:35:35,797 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-140792676] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,799 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,799 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:35:35,799 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:35:35,801 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,807 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1729144813] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,812 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,813 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,813 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,813 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,813 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:35:35,813 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,813 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:35,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 11:35:35,814 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,814 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:35,814 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:35,814 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:35,814 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,815 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,818 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:35,819 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@653c164e} 11:35:35,819 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,819 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,819 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,820 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,820 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,820 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,819 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 11:35:35,821 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:35,821 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:35,824 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:35,824 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:35,824 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:35,824 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 4.977 ms, resetting timer for the next recovery snapshot 11:35:35,824 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 11:35:35,829 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,829 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 8.847 ms, resetting timer for the next recovery snapshot 11:35:35,830 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,830 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#1738980947 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:35,830 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,830 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#1738980947] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,831 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:35,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:35,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1533855015] 11:35:35,831 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,831 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1533855015] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,832 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1461140067 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 11:35:35,833 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1461140067] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,833 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,835 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,835 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,835 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,835 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:35:35,835 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 11:35:35,836 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 11:35:35,836 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 11:35:35,838 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,840 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:35:35,840 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 11:35:35,840 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,840 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,840 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,840 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 11:35:35,840 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 11:35:35,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,842 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,842 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,844 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,844 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:35,844 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,845 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,845 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,845 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,845 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,845 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,848 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 3.121 ms, resetting timer for the next recovery snapshot 11:35:35,852 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,852 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,852 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:35:35,852 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 11:35:35,853 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,853 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 11:35:35,853 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 11:35:35,853 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,853 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,853 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,856 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 3.509 ms, resetting timer for the next recovery snapshot 11:35:35,866 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [9024544587302157838] 11:35:35,866 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:35:35,866 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 11:35:35,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,867 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,867 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,867 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 11:35:35,868 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:35:35,868 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 11:35:35,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:35,872 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:35:35,874 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:35,874 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:35,900 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,903 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,906 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 11:35:35,919 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:35:35,920 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:35:35,926 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-738234160] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,933 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:35:35,934 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 11:35:35,934 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 11:35:35,934 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:35,936 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@486cc7d} 11:35:35,936 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 11:35:35,937 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,937 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:35,937 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 11:35:35,941 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:35,944 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 11:35:35,946 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 11:35:35,947 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-815210948] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,947 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,948 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1088440461 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 11:35:35,948 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1088440461] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,948 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,951 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,951 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1587728518] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,951 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1289359884] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,952 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,952 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1376887461 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 11:35:35,952 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1376887461] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,953 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:35:35,953 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:35:35,960 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,960 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:35:35,960 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:35,961 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 11:35:35,961 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:35,962 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,963 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,963 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,963 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,964 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4a220b79} 11:35:35,965 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,966 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,966 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,966 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,966 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,967 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,967 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,967 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,969 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 11:35:35,969 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 3.185 ms, resetting timer for the next recovery snapshot 11:35:35,969 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:35,969 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:35,971 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 11:35:35,971 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,972 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,973 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,974 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-2061939781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:35:35,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 11:35:35,974 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,974 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#-1354708682 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:35,974 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 11:35:35,974 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#-1354708682] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,976 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,976 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,976 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,978 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 11:35:35,979 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#2009586757] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,980 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#506988338 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 11:35:35,980 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#506988338] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:35,980 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:35,980 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 4.248 ms, resetting timer for the next recovery snapshot 11:35:35,981 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,981 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,981 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,981 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,983 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 11:35:35,983 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 11:35:35,985 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:35,985 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:35,986 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:35,986 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:35,986 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,986 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,986 AM [cluster-test-shard-dispatcher-31] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,987 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,987 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,987 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:35:35,987 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 11:35:35,987 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,989 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:35,989 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:35,989 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:35,992 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 2.878 ms, resetting timer for the next recovery snapshot 11:35:35,994 AM [cluster-test-shard-dispatcher-31] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 7.795 ms, resetting timer for the next recovery snapshot 11:35:35,995 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:35:35,997 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 11:35:35,997 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:35:35,997 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 11:35:35,997 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 11:35:35,997 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,997 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,998 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,998 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:35,998 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 11:35:35,998 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:35:35,998 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 11:35:35,999 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 11:35:35,998 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:35,999 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 11:35:35,999 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 11:35:35,999 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 11:35:35,999 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:36,0 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:36,10 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:35:36,25 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-6617904095885666069] 11:35:36,25 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:35:36,29 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:36,29 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:35:36,31 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:36,31 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:36,51 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:36,55 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:36,61 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 11:35:36,66 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-648668595] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,71 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:35:36,72 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:35:36,73 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:35:36,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,74 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:36,75 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5b45eeaa} 11:35:36,75 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 11:35:36,75 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:36,76 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:36,76 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,79 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:36,82 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 11:35:36,87 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1941444984] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,88 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 11:35:36,91 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1737591474 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,91 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1737591474] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,91 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:36,90 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:36,93 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 11:35:36,95 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 11:35:36,95 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:36,95 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,97 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:36,98 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:36,98 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:36,98 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:36,99 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:35:36,100 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 11:35:36,101 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:35:36,101 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:36,102 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#1182959107 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,102 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 11:35:36,103 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:36,103 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:36,103 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:36,104 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:35:36,104 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,104 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,104 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:36,105 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@3cbce96} 11:35:36,105 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:36,105 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,105 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:36,105 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:36,105 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:36,107 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 11:35:36,107 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1436695817] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,107 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1497628392] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,107 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#1182959107] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,107 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#436514742] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,107 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:36,107 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:36,107 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:36,108 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#1595500199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,108 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:36,108 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#1595500199 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,107 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 4.430 ms, resetting timer for the next recovery snapshot 11:35:36,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,110 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-537724507] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,110 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 11:35:36,110 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 11:35:36,111 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-368269854 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,112 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-368269854] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:36,112 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 11:35:36,112 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 4.685 ms, resetting timer for the next recovery snapshot 11:35:36,112 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:36,112 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,112 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,113 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:36,113 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:35:36,113 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 11:35:36,113 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:36,113 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:36,113 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 11:35:36,113 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 11:35:36,113 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 11:35:36,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:35:36,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 11:35:36,116 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 11:35:36,116 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,116 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,116 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 11:35:36,117 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 11:35:36,117 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 11:35:36,117 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:36,117 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:36,117 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:36,120 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 3.311 ms, resetting timer for the next recovery snapshot 11:35:36,121 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:35:36,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:35:36,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:35:36,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:36,123 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:36,125 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,125 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:36,126 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:36,126 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,126 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:36,126 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,126 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:36,126 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:36,126 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:36,126 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,126 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,126 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 11:35:36,127 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:36,127 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 11:35:36,127 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 11:35:36,128 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 11:35:36,128 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 11:35:36,128 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 11:35:36,131 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 3.144 ms, resetting timer for the next recovery snapshot 11:35:36,135 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:35:36,135 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 11:35:36,135 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,135 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,135 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 11:35:36,136 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:36,136 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 11:35:36,136 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 11:35:36,738 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:35:36,738 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:35:36,740 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,741 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,742 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,742 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 11:35:36,742 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,756 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:36,756 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,757 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,884 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 11:35:36,885 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:35:36,885 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,885 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,885 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:36,886 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 11:35:36,891 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 11:35:36,891 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:35:36,891 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,891 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,891 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:36,892 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 11:35:36,899 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:35:36,902 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:35:36,912 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:35:36,914 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:35:36,914 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:36,914 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:36,914 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 11:35:36,914 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:35:36,914 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2905321b 11:35:36,915 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 11:35:36,914 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 11:35:36,915 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 11:35:36,915 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:35:36,915 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:36,915 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:36,915 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 11:35:36,915 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ad5c941 11:35:36,915 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 11:35:36,916 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37baaa5c 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:36,917 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:36,918 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,918 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:36,918 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:36,918 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 11:35:36,919 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 11:35:36,919 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f1d62c1 11:35:36,919 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 11:35:36,919 AM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 11:35:36,919 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 11:35:36,920 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 11:35:36,920 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33853873 11:35:36,921 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 11:35:36,921 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 11:35:36,922 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31e15a9d 11:35:36,922 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 11:35:36,922 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 11:35:36,936 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:35:36,936 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:36,936 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:35:36,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:36,936 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 11:35:36,936 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 11:35:36,937 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 11:35:36,937 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 11:35:36,949 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 11:35:36,949 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:35:36,949 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,949 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,949 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:36,950 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 11:35:36,952 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:35:36,960 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:35:36,961 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:35:36,961 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:36,961 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:36,961 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 11:35:36,961 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3412306b 11:35:36,961 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 11:35:36,964 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25b110ff 11:35:36,964 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:36,964 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 11:35:36,964 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 11:35:36,967 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 11:35:36,968 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 11:35:36,968 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 11:35:36,968 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a0540bb 11:35:36,968 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:36,974 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:35:36,974 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:36,974 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 11:35:36,974 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 11:35:36,974 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:36,974 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 11:35:36,974 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:36,989 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 11:35:36,990 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:35:36,990 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:36,990 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:36,990 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:36,990 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 11:35:36,997 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:35:37,5 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:35:37,6 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:35:37,6 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,6 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,6 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 11:35:37,6 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@302da231 11:35:37,6 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 11:35:37,9 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 11:35:37,9 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 11:35:37,9 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7fe030c3 11:35:37,9 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,10 AM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 11:35:37,13 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@773f67b 11:35:37,13 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,13 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 11:35:37,13 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 11:35:37,16 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:35:37,16 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,16 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 11:35:37,16 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 11:35:37,16 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,17 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 11:35:37,17 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:37,17 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 11:35:37,17 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:37,17 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 11:35:37,17 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:37,18 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,18 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,21 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 11:35:37,21 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 11:35:37,21 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 11:35:37,22 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:35:37,22 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:35:37,24 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:35:37,25 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:35:37,25 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:35:37,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:35:37,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:37,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:35:37,28 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:37,28 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:37,28 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:37,30 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:37,47 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 11:35:37,55 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:35:37,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 11:35:37,58 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,58 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:37,58 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,58 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,59 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,59 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:35:37,59 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-347157027] 11:35:37,59 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-347157027] 11:35:37,59 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,60 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57b5e218 11:35:37,60 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c8d9359 11:35:37,60 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c8d9359 11:35:37,60 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1729144813] 11:35:37,61 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:35:37,61 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:37,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:37,62 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@179f1da3 11:35:37,62 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@629325d9 11:35:37,62 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@629325d9 11:35:37,62 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,63 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1cfb40df 11:35:37,63 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1729144813] 11:35:37,63 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:35:37,63 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1533855015] 11:35:37,63 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1533855015] 11:35:37,64 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,64 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@54f44031 11:35:37,64 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-738234160] 11:35:37,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5dbc11e2 11:35:37,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5dbc11e2 11:35:37,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,64 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ee967e7 11:35:37,65 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:37,65 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-815210948] 11:35:37,65 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-815210948] 11:35:37,65 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cc21da9 11:35:37,65 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-738234160] 11:35:37,65 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:35:37,65 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1587728518] 11:35:37,65 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1587728518] 11:35:37,65 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1289359884] 11:35:37,66 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:35:37,66 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-2061939781] 11:35:37,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-2061939781] 11:35:37,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1289359884] 11:35:37,66 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 11:35:37,66 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#2009586757] 11:35:37,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#2009586757] 11:35:37,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-648668595] 11:35:37,66 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:37,67 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1941444984] 11:35:37,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1941444984] 11:35:37,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-648668595] 11:35:37,67 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:35:37,67 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1497628392] 11:35:37,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1497628392] 11:35:37,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1436695817] 11:35:37,67 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:35:37,67 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#436514742] 11:35:37,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#436514742] 11:35:37,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1436695817] 11:35:37,68 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:35:37,68 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-537724507] 11:35:37,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-537724507] 11:35:37,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,68 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:37,68 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,70 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47af9751 11:35:37,71 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,71 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e696a82 11:35:37,71 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b051e0f 11:35:37,72 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,73 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b1d85f 11:35:37,73 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b1d85f 11:35:37,73 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,73 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,74 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,74 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@102f6026 11:35:37,78 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7039a593 11:35:37,81 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,88 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d910def 11:35:37,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,92 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 11:35:37,92 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 11:35:37,92 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,92 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,93 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,93 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@476cedda 11:35:37,93 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 11:35:37,93 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 11:35:37,93 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 11:35:37,93 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 11:35:37,93 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,93 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 11:35:37,93 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,93 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,93 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,94 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,94 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5486ca57 11:35:37,94 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 11:35:37,94 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48d8863c 11:35:37,94 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 11:35:37,94 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:37,94 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,94 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@576aa493 11:35:37,94 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 11:35:37,94 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:37,94 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,94 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,94 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,94 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b9223a4 11:35:37,94 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77c88199 11:35:37,94 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 11:35:37,94 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,98 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 11:35:37,98 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 11:35:37,98 AM [cluster-test-shard-dispatcher-22] [INFO] AbstractLeader - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 11:35:37,98 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 11:35:37,99 AM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 11:35:37,99 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 11:35:37,99 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,99 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,99 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,99 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@503c2498 11:35:37,99 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 11:35:37,99 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:37,99 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,100 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,100 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,100 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@157e2945 11:35:37,100 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 11:35:37,100 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:37,100 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 11:35:37,100 AM [cluster-test-shard-dispatcher-34] [INFO] AbstractLeader - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 11:35:37,102 AM [cluster-test-shard-dispatcher-21] [INFO] AbstractLeader - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 11:35:37,102 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f83fa68 11:35:37,102 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 11:35:37,102 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 11:35:37,104 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 11:35:37,104 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 11:35:37,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,104 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5021d6b8 11:35:37,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,105 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,105 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@275b9c2 11:35:37,105 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@41b9cac4 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 11:35:37,105 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 11:35:37,105 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,105 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,105 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,105 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c147107 11:35:37,106 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,106 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 11:35:37,106 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@114ebcae 11:35:37,106 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 11:35:37,106 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 11:35:37,106 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,106 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,106 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,106 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 11:35:37,106 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57e20275 11:35:37,106 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:37,106 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1824f65f 11:35:37,106 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 11:35:37,107 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 11:35:37,109 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 11:35:37,109 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 11:35:37,109 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 11:35:37,121 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,172 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,223 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,274 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,324 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,414 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:37,414 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4179a5e0 11:35:37,414 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 352.0 ms 11:35:37,414 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 11:35:37,415 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 11:35:37,415 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 11:35:37,415 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 355.2 ms 11:35:37,415 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 351.3 ms 11:35:37,415 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 11:35:37,416 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 11:35:37,416 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ea3477 11:35:37,416 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49a1b665 11:35:37,416 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,422 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 11:35:37,422 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 11:35:37,422 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 11:35:37,422 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 349.7 ms 11:35:37,422 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 11:35:37,422 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 11:35:37,422 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1814e0d2 11:35:37,422 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:37,425 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 11:35:37,426 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1729144813] 11:35:37,427 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:35:37,427 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:37,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:37,427 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 11:35:37,427 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 11:35:37,427 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 11:35:37,427 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:35:37,427 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:35:37,428 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:35:37,428 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:35:37,428 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:35:37,428 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:35:37,428 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:37,428 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:35:37,429 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:37,430 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:37,430 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:37,430 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:37,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 11:35:37,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 11:35:37,431 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 11:35:37,431 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 11:35:37,431 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 11:35:37,432 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 11:35:37,433 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:35:37,433 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 11:35:37,433 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:35:37,433 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 11:35:37,433 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,433 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:37,433 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,433 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,441 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,441 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@32e581f4 11:35:37,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@32e581f4 11:35:37,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,442 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60578a91 11:35:37,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60578a91 11:35:37,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,442 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1c156dcb 11:35:37,442 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@13290458 11:35:37,442 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f480832 11:35:37,442 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d58ffb1 11:35:37,442 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f480832 11:35:37,442 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,443 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,443 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,441 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 11:35:37,443 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@22ad3c72 11:35:37,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@22ad3c72 11:35:37,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:37,444 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 11:35:37,444 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@17fd2744 11:35:37,452 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@faabc51 11:35:37,453 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57061463 11:35:37,458 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1fe40c18 11:35:37,458 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f386b54 11:35:37,459 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4998ae09 11:35:37,463 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7157ca55 11:35:37,465 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e8693ac 11:35:37,465 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1df93a33 11:35:37,466 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 11:35:37,466 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:35:37,466 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,471 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,471 AM [cluster-test-shard-dispatcher-34] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 11:35:37,472 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,472 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:35:37,472 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 11:35:37,472 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,472 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,472 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@41213ff2 11:35:37,472 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 11:35:37,472 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:35:37,472 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,472 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,472 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,472 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,472 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@647ea9af 11:35:37,472 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@32f5e870 11:35:37,473 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 11:35:37,473 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 11:35:37,475 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 11:35:37,475 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 11:35:37,475 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:35:37,475 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,475 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,475 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 11:35:37,475 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 11:35:37,475 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 11:35:37,476 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b8916fd 11:35:37,476 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 11:35:37,477 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 11:35:37,484 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,485 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:35:37,485 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-347157027] 11:35:37,486 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-347157027] 11:35:37,486 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1729144813] 11:35:37,486 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:35:37,486 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:37,486 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:35:37,486 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,487 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,487 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:35:37,487 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,487 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,488 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,488 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#270786104] 11:35:37,488 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:35:37,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,488 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39b38405 11:35:37,488 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:35:37,488 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 11:35:37,488 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 11:35:37,488 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 11:35:37,488 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,488 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@54645157 11:35:37,488 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,488 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 11:35:37,488 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,488 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@565a06df 11:35:37,488 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 11:35:37,489 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@36e9ca95 11:35:37,489 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 11:35:37,489 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 11:35:37,489 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:37,489 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1729144813] 11:35:37,489 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:35:37,489 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1533855015] 11:35:37,489 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1533855015] 11:35:37,489 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-738234160] 11:35:37,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:37,490 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-815210948] 11:35:37,490 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-815210948] 11:35:37,490 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-738234160] 11:35:37,491 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:35:37,491 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 11:35:37,491 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1587728518] 11:35:37,491 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 11:35:37,492 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4747b98c 11:35:37,492 AM [cluster-test-shard-dispatcher-24] [INFO] AbstractLeader - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 11:35:37,492 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 11:35:37,492 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 11:35:37,492 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 11:35:37,492 AM [cluster-test-shard-dispatcher-20] [INFO] AbstractLeader - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 11:35:37,492 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@204bd919 11:35:37,492 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1103ec69 11:35:37,492 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e8c2360 11:35:37,492 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 11:35:37,492 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 11:35:37,492 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1587728518] 11:35:37,493 AM [cluster-test-shard-dispatcher-30] [INFO] AbstractLeader - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 11:35:37,493 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1289359884] 11:35:37,493 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:35:37,493 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-2061939781] 11:35:37,493 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-2061939781] 11:35:37,493 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1289359884] 11:35:37,493 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 11:35:37,493 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#2009586757] 11:35:37,494 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#2009586757] 11:35:37,495 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-648668595] 11:35:37,495 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:37,495 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 11:35:37,495 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,495 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,496 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1941444984] 11:35:37,496 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,497 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 11:35:37,497 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,497 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,497 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,497 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@208fe451 11:35:37,497 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 11:35:37,498 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 11:35:37,498 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 11:35:37,498 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 11:35:37,498 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 11:35:37,496 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 11:35:37,498 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,498 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,496 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 11:35:37,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,499 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 11:35:37,499 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:37,499 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 11:35:37,496 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@455791c1 11:35:37,496 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1941444984] 11:35:37,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 11:35:37,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1251c12a 11:35:37,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 11:35:37,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63a31a74 11:35:37,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 11:35:37,499 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-648668595] 11:35:37,499 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:35:37,499 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1497628392] 11:35:37,499 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1497628392] 11:35:37,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1436695817] 11:35:37,500 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:35:37,500 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#436514742] 11:35:37,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#436514742] 11:35:37,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1436695817] 11:35:37,501 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:35:37,501 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-537724507] 11:35:37,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-537724507] 11:35:37,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1256617334] 11:35:37,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:37,502 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,502 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#532300995] 11:35:37,502 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:37,503 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 11:35:37,503 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 11:35:37,503 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 11:35:37,504 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 11:35:37,504 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Stopping Shard 11:35:37,504 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal closed 11:35:37,504 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal closed 11:35:37,504 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:35:37,505 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:35:37,505 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:37,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 11:35:37,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 11:35:37,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:35:37,506 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 11:35:37,507 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal closed 11:35:37,507 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 11:35:37,507 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal closed 11:35:37,508 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:35:37,508 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:35:37,510 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:37,510 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:37,519 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:37,519 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:37,519 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:37,519 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:37,524 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:35:37,538 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:37,539 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 11:35:37,539 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 11:35:37,539 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:35:37,541 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 11:35:37,541 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Stopping Shard 11:35:37,542 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal closed 11:35:37,542 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal closed 11:35:37,543 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:35:37,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:35:37,544 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:37,544 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 11:35:37,544 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 11:35:37,544 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:35:37,545 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 11:35:37,546 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 11:35:37,546 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal closed 11:35:37,546 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal closed 11:35:37,546 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:35:37,546 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:35:37,548 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:37,548 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:37,757 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:35:38,527 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-6617904095885666069] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:35:38,567 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [9024544587302157838] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:35:38,570 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:38,571 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:38,579 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:35:38.581] [cluster-test-pekko.actor.internal-dispatcher-7] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:38,586 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 11:35:38,586 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 11:35:38,587 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 11:35:38,587 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:35:38,587 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49cfbe40 11:35:38,587 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1dbfc07f 11:35:38,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@d907795 11:35:38,587 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a8dc2a7 11:35:38,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@d907795 11:35:38,587 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a8dc2a7 11:35:38,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:38,587 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:39,645 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 11:35:39,645 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 11:35:39,645 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.059 s 11:35:39,645 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.059 s 11:35:39,646 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 11:35:39,646 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 11:35:39,647 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 11:35:39,647 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Stopping Shard 11:35:39,647 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal closed 11:35:39,647 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal closed 11:35:39,648 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:35:39,648 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:35:39,649 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 11:35:39,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 11:35:39,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 11:35:39,651 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:35:39,651 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3889ec61 11:35:39,651 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3889ec61 11:35:39,651 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:39,651 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@27260bee 11:35:39,651 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@27260bee 11:35:39,651 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 11:35:39,651 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e11ecce 11:35:39,651 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a9866f2 11:35:40,716 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 11:35:40,716 AM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.066 s 11:35:40,717 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 11:35:40,718 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 11:35:40,719 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal closed 11:35:40,726 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 11:35:40,726 AM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.076 s 11:35:40,726 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 11:35:40,727 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 11:35:40,727 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal closed 11:35:40,728 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:35:40,728 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:35:40,730 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:40,731 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:40,738 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:35:40.741] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:40,766 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:35:40,786 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-2169816869150843214] 11:35:40,787 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:35:40,791 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:35:40,791 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:35:40,792 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:40,792 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:40,811 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:40,812 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:35:40,812 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:35:40,812 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:35:40,817 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:40,819 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 11:35:40,826 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#389136938] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:35:40,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:35:40,833 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:35:40,833 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:40,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 11:35:40,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:40,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:40,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 11:35:40,840 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 11:35:40,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5f17220d} 11:35:40,842 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 11:35:40,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:40,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:40,843 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:40,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:40,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:40,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 11:35:40,844 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 11:35:40,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 11:35:40,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#1965437285] 11:35:40,844 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:40,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-1182372234 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:40,845 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:40,845 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#-1182372234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,846 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:40,846 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#1965437285] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,847 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1787112369 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 11:35:40,847 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:40,847 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1787112369] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,848 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:40,850 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:40,853 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:40,853 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:40,854 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:40,855 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:40,858 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:40,858 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:40,859 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,859 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,859 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 11:35:40,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,860 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 11:35:40,860 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:35:40,860 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 11:35:40,860 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:35:40,860 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 11:35:40,863 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 11:35:40,863 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:40,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,863 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:40,864 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:40,864 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:40,864 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:40,864 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:40,864 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 11:35:40,864 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7904bb3e 11:35:40,864 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 11:35:40,864 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 11:35:40,864 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:40,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,864 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:40,864 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:40,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:40,865 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:40,865 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:40,865 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 11:35:40,865 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c547dcb 11:35:40,865 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 11:35:40,865 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:40,892 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#389136938] 11:35:40,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:40,893 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:40,893 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:40,897 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:40,903 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 11:35:40,905 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1394288764] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,906 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:35:40,907 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:35:40,908 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:35:40,908 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:40,908 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:40,908 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:40,909 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:40,909 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:40,909 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 11:35:40,910 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@2f69218d} 11:35:40,910 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 11:35:40,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:40,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:40,911 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:40,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:40,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651] 11:35:40,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:40,912 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:40,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:40,912 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:40,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#952456594] 11:35:40,912 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:40,913 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:40,913 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-610455766 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:40,914 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,914 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#224932333 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:40,914 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#952456594] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,914 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-610455766] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,914 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:40,914 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#224932333] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:40,915 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:40,915 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:40,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:40,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:40,916 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:40,916 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:40,920 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:40,921 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:35:40,921 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 11:35:40,921 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 11:35:40,921 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:40,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:35:40,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 11:35:40,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,921 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 11:35:40,923 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 11:35:40,923 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:40,923 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,923 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,923 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:40,924 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:40,924 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:40,924 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:40,924 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:40,924 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 11:35:40,924 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@44f48d03 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:40,924 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:40,924 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:40,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:40,924 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:40,924 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 11:35:40,924 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ecb710d 11:35:40,925 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 11:35:40,925 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:40,961 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1394288764] 11:35:40,961 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:35:40,961 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651] 11:35:40,973 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:35:40,988 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [6438691529423924332] 11:35:40,988 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:35:40,991 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:40,991 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:35:40,992 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:40,992 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:41,3 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,5 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,12 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 11:35:41,17 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:41,38 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 11:35:41,40 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 11:35:41,41 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,43 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,45 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 11:35:41,46 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:35:41,47 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:35:41,48 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:35:41,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@e531d7f} 11:35:41,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:41,49 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1250223016] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:41,51 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:35:41,52 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:35:41,56 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:35:41,59 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 11:35:41,60 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 11:35:41,65 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@6700132e} 11:35:41,68 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:41,82 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:35:41,87 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:41,87 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:41,87 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:41,87 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:41,87 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:41,89 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:41,99 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [1726726737994632858] 11:35:41,100 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:35:41,102 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:41,102 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:35:41,104 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:41,104 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:41,120 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,122 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,126 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 11:35:41,136 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:41,136 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:35:41,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:35:41,162 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:35:41,163 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:35:41,165 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:41,165 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 11:35:41,168 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:41,168 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:41,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@19fa2c66} 11:35:41,169 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 11:35:41,169 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,171 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:41,172 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:41,175 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 11:35:41,176 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:35:41,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:35:41,177 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-879283811] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:41,179 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:35:41,179 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:41,179 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:41,182 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 11:35:41,183 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@173dd129} 11:35:41,183 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 11:35:41,184 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:41,812 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:35:41,813 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:35:41,814 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:41,814 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 11:35:41,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:41,814 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 11:35:41,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:41,815 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:41,815 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 11:35:41,815 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:41,816 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:41,825 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:41,825 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:41,825 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:41,825 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:41,826 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:41,826 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:41,826 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:41,826 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:42,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,33 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:42,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:42,33 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:42,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,33 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:42,33 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:42,89 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:35:42,89 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 11:35:42,90 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,90 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 11:35:42,94 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,94 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551, primaryVersion=13] 11:35:42,98 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,100 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,101 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:42,102 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:42,103 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#627058685 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,103 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#627058685] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,103 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,104 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:42,104 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:42,104 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:42,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:42,106 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:42,109 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.104442743Z 11:35:42,109 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:42,110 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:35:42,110 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 11:35:42,110 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,110 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,110 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 11:35:42,110 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:42,113 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:42,113 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 11:35:42,113 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,113 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69c7f1be 11:35:42,113 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 11:35:42,113 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,113 AM [cluster-test-shard-dispatcher-22] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:42,117 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:35:42,117 AM [cluster-test-shard-dispatcher-22] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.114241686Z 11:35:42,118 AM [cluster-test-shard-dispatcher-22] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 11:35:42,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:42,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-952508201] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 11:35:42,118 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-952508201] 11:35:42,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 669.5 μs 11:35:42,119 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 11:35:42,119 AM [cluster-test-shard-dispatcher-22] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 11:35:42,119 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 141.8 μs 11:35:42,119 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-952508201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,120 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551 11:35:42,123 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 11:35:42,124 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,125 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 11:35:42,126 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:42,128 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:35:42,128 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 11:35:42,129 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,129 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4944426c 11:35:42,129 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] 11:35:42,129 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,129 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,130 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:42,130 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1250223016] 11:35:42,130 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:35:42,130 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 11:35:42,130 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:35:42,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 11:35:42,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 11:35:42,133 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,133 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651, primaryVersion=13] 11:35:42,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,135 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:42,135 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#808222003] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,136 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-844256797 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,136 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-844256797] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,137 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:42,137 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,138 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:35:42,138 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:42,139 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:42,141 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:42,141 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:42,142 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.138342979Z 11:35:42,145 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:42,145 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:35:42,146 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 11:35:42,146 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,146 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,146 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 11:35:42,146 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:35:42,148 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,149 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 11:35:42,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 11:35:42,149 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:42,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@61f42258 11:35:42,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 11:35:42,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:42,151 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.149541351Z 11:35:42,152 AM [cluster-test-shard-dispatcher-25] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 11:35:42,152 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:42,152 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1680116818] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 11:35:42,152 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1680116818] 11:35:42,152 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 523.5 μs 11:35:42,152 AM [cluster-test-shard-dispatcher-25] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 11:35:42,152 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 11:35:42,152 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 65.62 μs 11:35:42,152 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1680116818] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,154 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:35:42,156 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651 11:35:42,156 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 11:35:42,157 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,157 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 11:35:42,158 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:42,160 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3eaa652c 11:35:42,160 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:35:42,160 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:42,161 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1250223016] 11:35:42,161 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:35:42,161 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#808222003] 11:35:42,161 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#808222003] 11:35:42,162 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:35:42,162 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 11:35:42,163 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,163 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 11:35:42,163 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 11:35:42,163 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,163 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 11:35:42,165 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,165 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551, primaryVersion=13] 11:35:42,168 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:42,168 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:42,169 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,169 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-290410502 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:42,169 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:42,169 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-290410502] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,170 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:42,171 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:42,172 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:42,172 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:42,173 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 11:35:42,173 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:42,175 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:42,175 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:35:42,175 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,175 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 11:35:42,175 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,175 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 11:35:42,175 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 11:35:42,176 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.173180571Z 11:35:42,178 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:42,178 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:42,179 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,179 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 11:35:42,179 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@46271949 11:35:42,179 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 11:35:42,180 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,182 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.178957232Z 11:35:42,182 AM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 11:35:42,182 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:42,182 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#880037699] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 11:35:42,182 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#880037699] 11:35:42,182 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 588.8 μs 11:35:42,182 AM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 11:35:42,183 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 11:35:42,183 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 88.20 μs 11:35:42,183 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#880037699] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,184 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 11:35:42,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551 11:35:42,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 11:35:42,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:42,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 11:35:42,187 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:42,190 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,190 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:35:42,190 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,190 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] 11:35:42,190 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@273dd857 11:35:42,190 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:42,190 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,190 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,191 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-879283811] 11:35:42,191 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 11:35:42,191 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:35:42,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 11:35:42,191 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,191 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:35:42,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 11:35:42,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 11:35:42,193 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:35:42,193 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651, primaryVersion=13] 11:35:42,195 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,195 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 11:35:42,195 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1182572335] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,196 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-2054304990 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,196 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-2054304990] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,197 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,198 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 11:35:42,198 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:35:42,201 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.198316551Z 11:35:42,203 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 11:35:42,204 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 11:35:42,206 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:42,206 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:42,209 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 11:35:42,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:35:42,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 11:35:42,210 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,210 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,210 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 11:35:42,210 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 11:35:42,212 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,212 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 11:35:42,212 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:35:42,213 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 11:35:42,213 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e137d5d 11:35:42,213 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 11:35:42,213 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:42,216 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-21T11:35:42.213369521Z 11:35:42,216 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 11:35:42,216 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:35:42,216 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-887715444] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 11:35:42,217 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-887715444] 11:35:42,217 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 622.6 μs 11:35:42,217 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 11:35:42,217 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 11:35:42,217 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 116.2 μs 11:35:42,217 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-887715444] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,218 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 11:35:42,220 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-607514651 11:35:42,220 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 11:35:42,221 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,221 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 11:35:42,222 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 11:35:42,222 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:42,223 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 11:35:42,224 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 11:35:42,224 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,224 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a827712 11:35:42,224 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 11:35:42,224 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 11:35:42,225 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:35:42,225 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-879283811] 11:35:42,225 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:35:42,225 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1182572335] 11:35:42,225 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1182572335] 11:35:42,225 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#389136938] 11:35:42,225 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:42,225 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,225 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,226 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] 11:35:42,226 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:42,226 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,226 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,226 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] 11:35:42,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:42,226 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,226 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,228 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard ShardName{value=cars}, datastoreType Config 11:35:42,228 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#389136938] 11:35:42,229 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:42,229 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,230 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 11:35:42,231 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 11:35:42,232 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#389136938] 11:35:42,232 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:42,232 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,232 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,236 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] 11:35:42,236 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:42,236 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,236 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,236 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] 11:35:42,236 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:42,237 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,237 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,237 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard ShardName{value=cars}, datastoreType Config 11:35:42,237 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] 11:35:42,237 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:42,237 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,237 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,237 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 11:35:42,246 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2284cbbf 11:35:42,247 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1434bf29 11:35:42,247 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1434bf29 11:35:42,247 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 11:35:42,249 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@113b586a 11:35:42,249 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,250 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40d133d0 11:35:42,251 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 11:35:42,252 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,255 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 11:35:42,256 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:35:42,256 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,256 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,256 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:42,256 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f817481 11:35:42,256 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 11:35:42,257 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 1 - switching to Follower 11:35:42,257 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:35:42,266 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 11:35:42,266 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:42,266 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:42,266 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@551617b7 11:35:42,266 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:42,266 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 11:35:42,266 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,267 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 11:35:42,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,267 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,267 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c43219f 11:35:42,267 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 11:35:42,267 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 11:35:42,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@287ca0cf 11:35:42,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@967b4fd 11:35:42,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,269 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,269 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 22.76 ms 11:35:42,269 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,270 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,275 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,276 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 11:35:42,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] 11:35:42,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:42,276 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] 11:35:42,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#389136938] 11:35:42,277 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:42,277 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1658506551] 11:35:42,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] 11:35:42,278 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:42,278 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,278 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard ShardName{value=cars}, datastoreType Config 11:35:42,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] 11:35:42,278 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:42,278 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,278 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 11:35:42,281 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e3e694b 11:35:42,281 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e3e694b 11:35:42,281 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 11:35:42,281 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31b2922c 11:35:42,282 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6d7e2ba4 11:35:42,283 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c1bc919 11:35:42,284 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 11:35:42,289 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 11:35:42,289 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 11:35:42,289 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,289 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,289 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:42,289 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69f20a89 11:35:42,289 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 11:35:42,290 AM [cluster-test-shard-dispatcher-25] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 2 - switching to Follower 11:35:42,290 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 2 - updating term 11:35:42,297 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:42,297 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:42,297 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:42,297 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 11:35:42,297 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:42,297 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cfefe32 11:35:42,297 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,297 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,297 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 11:35:42,297 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,297 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@24b1adcd 11:35:42,297 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 11:35:42,297 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 11:35:42,298 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:42,298 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a016478 11:35:42,298 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 17.71 ms 11:35:42,298 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,298 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,298 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6923cc87 11:35:42,299 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,299 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,299 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 11:35:42,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] 11:35:42,300 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:42,300 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#31117702] 11:35:42,301 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,301 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 11:35:42,302 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 11:35:42,302 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 11:35:42,302 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,302 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26dd74b5 11:35:42,302 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72055785 11:35:42,302 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72055785 11:35:42,302 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 11:35:42,303 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 11:35:42,303 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e1032c5 11:35:42,304 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c5947e5 11:35:42,308 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 11:35:42,309 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:35:42,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,309 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:42,309 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6497ade1 11:35:42,309 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 11:35:42,310 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 3 - updating term 11:35:42,310 AM [cluster-test-shard-dispatcher-30] [INFO] AbstractLeader - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 3 - switching to Follower 11:35:42,320 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:35:42,320 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:42,320 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:42,320 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:42,320 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@979375a 11:35:42,320 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 11:35:42,320 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 11:35:42,320 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,320 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,320 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,321 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 11:35:42,321 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@61db82c3 11:35:42,321 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 11:35:42,322 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11eefe79 11:35:42,322 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,322 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,323 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,323 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 20.77 ms 11:35:42,323 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@14637185 11:35:42,323 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,323 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,323 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 11:35:42,324 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal closed 11:35:42,324 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:35:42,324 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:35:42,327 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 11:35:42,328 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-1727345394] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,328 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-783897278] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,328 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 11:35:42,329 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,329 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 11:35:42,329 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 11:35:42,330 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal closed 11:35:42,330 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:35:42,330 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:35:42,332 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:42,332 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:42,337 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:42,337 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:42,337 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:42,337 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:42,339 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:35:42,357 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 11:35:42,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 11:35:42,359 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 11:35:42,359 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cb7226d 11:35:42,359 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73597e7d 11:35:42,359 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73597e7d 11:35:42,359 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 11:35:42,362 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40fc03e6 11:35:42,363 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 11:35:42,368 AM [cluster-test-shard-dispatcher-31] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 5 11:35:42,368 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:35:42,368 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,368 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,368 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d51d4eb 11:35:42,368 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 11:35:42,369 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 11:35:42,369 AM [cluster-test-shard-dispatcher-25] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 4 - switching to Follower 11:35:42,374 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:35:42,374 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 11:35:42,374 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:42,374 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 11:35:42,374 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56dfa1c8 11:35:42,374 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 11:35:42,375 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:35:42,375 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:42,375 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 11:35:42,375 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 11:35:42,375 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@358de8f4 11:35:42,376 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 11:35:42,376 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:42,376 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@45c125f3 11:35:42,377 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 11:35:42,376 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 17.45 ms 11:35:42,377 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 11:35:42,377 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 11:35:42,378 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal closed 11:35:42,378 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:35:42,378 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:35:42,378 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 11:35:42,379 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 11:35:42,379 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 11:35:42,379 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,379 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#844535150] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:42,380 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 11:35:42,382 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal closed 11:35:42,383 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:35:42,383 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:35:42,384 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:42,384 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:42,832 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:35:43,352 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [1726726737994632858] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:35:43,400 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [6438691529423924332] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:35:43,401 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:43,402 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:43,404 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:35:43.406] [cluster-test-pekko.actor.internal-dispatcher-7] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:43,419 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 11:35:43,420 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 11:35:43,420 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 11:35:43,420 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:35:43,420 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@568cb3c6 11:35:43,421 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ae17632 11:35:43,421 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1249088e 11:35:43,421 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ae17632 11:35:43,421 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1249088e 11:35:43,421 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 11:35:43,421 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Stopping Shard 11:35:43,422 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal closed 11:35:47,935 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 11:35:47,935 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 11:35:47,935 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:47,935 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:35:47,935 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:47,935 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:47,935 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:49,781 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:50,987 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:35:52,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 11:35:52,571 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.151 s 11:35:52,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 11:35:52,572 AM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 11:35:52,572 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 5 11:35:52,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:52,573 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 11:35:52,573 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 11:35:52,573 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 11:35:52,574 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal closed 11:35:52,574 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:35:52,575 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:35:52,575 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 11:35:52,576 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 11:35:52,576 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 11:35:52,576 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:35:52,576 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@91f2d03 11:35:52,576 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37e14c63 11:35:52,576 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@91f2d03 11:35:52,577 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Stopping Shard 11:35:52,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5db3ee8c 11:35:52,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5db3ee8c 11:35:52,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 11:35:52,577 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal closed 11:35:56,151 AM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 11:35:56,152 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:35:56,152 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 11:35:56,152 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.576 s 11:35:56,152 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 11:35:56,152 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 11:35:56,152 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 11:35:56,152 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 11:35:56,153 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 11:35:56,153 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal closed 11:35:56,154 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:35:56,154 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:35:56,156 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:56,156 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:56,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:35:56.163] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:56,187 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:56,213 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [432160473839307484] 11:35:56,214 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:35:56,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:35:56,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:35:56,220 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:56,220 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:56,240 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:35:56,240 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:35:56,240 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:35:56,241 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,243 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,246 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 11:35:56,253 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1403293535] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,253 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:35:56,254 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:35:56,257 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:35:56,257 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 11:35:56,258 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 11:35:56,259 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:56,259 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,259 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,260 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:35:56,262 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@37960c56} 11:35:56,262 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 11:35:56,262 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,264 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,266 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:56,266 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,267 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#236169244] 11:35:56,268 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 11:35:56,268 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#236169244] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,269 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,270 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 11:35:56,271 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#1826721513] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,272 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#1826721513 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 11:35:56,273 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 11:35:56,275 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:35:56,275 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:35:56,276 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,276 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,276 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1592539314] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,277 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 11:35:56,279 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,279 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#789718778 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 11:35:56,279 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,279 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#789718778] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,280 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 11:35:56,281 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,282 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,282 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:35:56,282 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,282 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,282 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:35:56,282 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 11:35:56,282 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 11:35:56,282 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:56,282 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 11:35:56,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 11:35:56,283 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:56,283 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,283 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,284 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 11:35:56,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,284 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,285 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@6cc54fcc} 11:35:56,285 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 11:35:56,286 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:56,286 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#384469969] 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 11:35:56,287 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,287 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#1910011106] 11:35:56,287 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,288 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,288 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:56,288 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 11:35:56,288 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#384469969] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,288 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#1910011106] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,288 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#1537378611 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,288 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#1537378611] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,288 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:35:56,288 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 11:35:56,288 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,288 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:56,288 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 11:35:56,289 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-373858970 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 11:35:56,290 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-373858970] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,293 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,295 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:56,295 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:56,297 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,298 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,298 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,299 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,300 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:56,302 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,302 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:35:56,302 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 11:35:56,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,302 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 11:35:56,303 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:56,303 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 11:35:56,306 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,307 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,307 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,307 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:35:56,307 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 11:35:56,308 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 11:35:56,308 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:56,308 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 11:35:56,314 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5723284641946130884] 11:35:56,314 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:35:56,318 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:56,318 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:35:56,326 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:56,326 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:56,331 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,334 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,359 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 11:35:56,373 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:35:56,374 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:35:56,374 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-101780700] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,419 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:35:56,420 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 11:35:56,422 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 11:35:56,422 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:35:56,423 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@22ac491f} 11:35:56,426 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 11:35:56,427 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,427 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:56,427 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 11:35:56,430 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,434 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:35:56,434 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 11:35:56,437 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:56,438 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 11:35:56,439 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:56,439 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,439 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:56,439 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,439 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:56,440 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1044072704] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,440 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 11:35:56,440 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1680573976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,440 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-242929278 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 11:35:56,440 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-242929278] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,440 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,440 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:35:56,441 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:35:56,441 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#1971625826] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,441 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 11:35:56,442 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#722114583 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 11:35:56,442 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#722114583] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,442 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 11:35:56,442 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,444 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,445 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 11:35:56,445 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,446 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,446 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,449 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,450 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:35:56,451 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,451 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 11:35:56,451 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,451 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:35:56,451 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 11:35:56,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 11:35:56,451 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 11:35:56,452 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,452 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,452 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:56,452 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:56,453 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 11:35:56,454 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 11:35:56,454 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:35:56,454 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 11:35:56,455 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 11:35:56,455 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:56,455 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,455 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,456 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 11:35:56,456 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@528676dd} 11:35:56,456 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 11:35:56,457 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:56,457 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-2-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#1459251917] 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:56,458 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-2-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-280382116] 11:35:56,459 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#1459251917] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,459 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-280382116] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,459 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:56,460 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-2107288299 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 11:35:56,460 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-2107288299] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,460 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,461 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:56,461 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:56,462 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#841592535 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 11:35:56,462 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#841592535] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,462 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,463 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,463 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,464 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:56,464 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,464 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,467 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,467 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:35:56,467 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 11:35:56,468 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,468 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,468 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 11:35:56,468 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,468 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:56,468 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 11:35:56,468 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,468 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,468 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,468 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 11:35:56,468 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:35:56,468 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 11:35:56,468 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,469 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:56,469 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 11:35:56,471 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:35:56,483 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [687338696277365272] 11:35:56,484 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:35:56,486 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:35:56,486 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:35:56,493 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:56,493 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:56,500 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,502 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,505 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 11:35:56,512 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1760713306] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,515 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:35:56,515 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:35:56,520 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:35:56,520 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 11:35:56,520 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 11:35:56,523 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:35:56,524 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@465769d0} 11:35:56,524 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 11:35:56,524 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,526 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:56,528 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 11:35:56,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:56,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 11:35:56,530 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 11:35:56,531 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#1563676466] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,532 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 11:35:56,532 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,532 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1403028330 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 11:35:56,532 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1403028330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,533 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 11:35:56,534 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 11:35:56,535 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 11:35:56,535 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:35:56,536 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:35:56,536 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:35:56,537 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,537 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,538 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:35:56,538 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 11:35:56,538 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 11:35:56,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@355ef7e} 11:35:56,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:56,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 11:35:56,540 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,542 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:56,543 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,543 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1475641021 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 11:35:56,545 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:56,545 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:35:56,545 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 11:35:56,546 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,546 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-1173791853 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 11:35:56,547 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:35:56,547 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:35:56,547 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 11:35:56,549 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:56,549 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 11:35:56,549 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,549 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:56,549 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,549 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:56,550 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,550 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,552 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 11:35:56,553 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-898783034] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,553 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1892517254] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,553 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-420973056] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,553 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1475641021] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,553 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-1173791853] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,553 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:35:56,553 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,553 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:56,553 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,553 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:56,554 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:56,554 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:35:56,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 11:35:56,554 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,554 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,554 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#1484233712] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,554 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:56,555 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:56,555 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 11:35:56,555 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:56,555 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#-1668629176] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:56,555 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 11:35:56,556 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:56,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,557 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:56,557 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 11:35:56,558 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:56,558 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 11:35:56,560 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#-1668629176 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 11:35:56,561 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,561 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,562 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:56,563 AM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:56,565 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,565 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,565 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,565 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 11:35:56,566 AM [cluster-test-shard-dispatcher-33] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:56,566 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:56,566 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:35:56,566 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 11:35:56,566 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:56,566 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 11:35:56,566 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:35:56,566 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 11:35:56,566 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,566 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,566 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 11:35:56,566 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:56,566 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:56,567 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 11:35:56,636 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 11:35:56,636 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:56,636 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,636 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,636 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 11:35:56,636 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 11:35:56,657 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 11:35:56,657 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:56,657 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,657 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,657 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 11:35:56,657 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 11:35:56,658 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 11:35:56,658 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:56,658 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,658 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,658 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 11:35:56,658 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 11:35:56,696 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 11:35:56,696 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:35:56,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:56,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:56,697 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 11:35:56,697 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 11:35:56,966 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 11:35:57,46 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 11:35:57,56 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 11:35:57,66 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 11:35:57,234 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:35:57,234 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:57,235 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:57,236 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,236 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:57,245 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,245 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,246 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,317 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 11:35:57,320 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,320 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,329 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:57,329 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:35:57,329 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 11:35:57,329 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@13e3298d 11:35:57,329 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 11:35:57,329 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 11:35:57,331 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 11:35:57,331 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27f3f7 11:35:57,332 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 11:35:57,332 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4bd9db6d 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:35:57,361 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:35:57,397 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 11:35:57,398 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 11:35:57,400 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,400 AM [cluster-test-shard-dispatcher-20] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,401 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,401 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,410 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:57,410 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:35:57,410 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 11:35:57,410 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 11:35:57,410 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d24d581 11:35:57,411 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 11:35:57,411 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:35:57,411 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:57,411 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:35:57,411 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 11:35:57,411 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21d01f40 11:35:57,411 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 11:35:57,411 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 11:35:57,412 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@685ad08a 11:35:57,412 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:35:57,412 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 11:35:57,412 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 11:35:57,412 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@650a8bfa 11:35:57,413 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 11:35:57,413 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5142b989 11:35:57,413 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 11:35:57,413 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:35:57,413 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f855dd1 11:35:57,426 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 11:35:57,428 AM [cluster-test-shard-dispatcher-20] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,429 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:35:57,439 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:35:57,439 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:35:57,439 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 11:35:57,439 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 11:35:57,439 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@191b42fd 11:35:57,439 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 11:35:57,439 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 11:35:57,439 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 11:35:57,439 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:35:57,440 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5bbad2f9 11:35:57,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 11:35:57,441 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 11:35:57,441 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6775ff55 11:35:57,441 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1403293535] 11:35:57,441 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 11:35:57,441 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 11:35:57,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 11:35:57,441 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:57,441 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:35:57,441 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:35:57,442 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-101780700] 11:35:57,442 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:57,442 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1680573976] 11:35:57,443 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1680573976] 11:35:57,443 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1760713306] 11:35:57,443 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:57,443 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1892517254] 11:35:57,443 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1892517254] 11:35:57,447 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-3}, datastoreType Config 11:35:57,447 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 11:35:57,450 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:35:57,450 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 11:35:57,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 11:35:57,461 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 11:35:57,461 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 11:35:57,461 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 11:35:57,463 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1403293535] 11:35:57,463 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:35:57,463 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:35:57,463 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:35:57,463 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-101780700] 11:35:57,464 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:35:57,464 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1680573976] 11:35:57,464 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1680573976] 11:35:57,464 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:35:57,465 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1760713306] 11:35:57,465 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 11:35:57,465 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:57,465 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1892517254] 11:35:57,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1892517254] with 27000 ms timeout 11:35:57,468 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Stopping Shard 11:35:57,468 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 11:35:57,468 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 11:35:57,468 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal closed 11:35:57,469 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1892517254] 11:35:57,512 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:35:57,515 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1760713306] 11:35:57,516 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:35:57,516 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 11:35:57,516 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:35:57,517 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 11:35:57,517 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 11:35:57,517 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:35:57,518 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 11:35:57,518 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 11:35:57,518 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 11:35:57,519 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 11:35:57,520 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:35:57,520 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:35:57,520 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 11:35:57,521 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 11:35:57,521 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,521 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 11:35:57,521 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:35:57,522 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 11:35:57,522 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 11:35:57,522 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 11:35:57,522 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 11:35:57,522 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 11:35:57,523 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:35:57,523 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:35:57,525 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 11:35:57,525 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:35:57,525 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:35:57,528 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 11:35:57,528 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 11:35:57,533 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:57,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:57,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:57,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:35:57,536 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:35:57,566 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:35:57,579 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-3230057587588849994] 11:35:57,580 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:35:57,584 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 11:35:57,584 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:35:57,585 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:35:57,585 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:35:57,615 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:57,617 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:57,620 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 11:35:57,631 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:35:57,631 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:35:57,634 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1859027857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,640 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:35:57,640 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 11:35:57,640 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 11:35:57,644 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4b4d742a} 11:35:57,645 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 11:35:57,645 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 11:35:57,645 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:57,646 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 11:35:57,646 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:57,647 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 11:35:57,647 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:35:57,647 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 11:35:57,648 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:57,650 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 11:35:57,651 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 11:35:57,653 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:35:57,653 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:35:57,654 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:35:57,654 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-863946025 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 11:35:57,655 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#879611252] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,655 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1476587336] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,655 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-863946025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,655 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:57,655 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:57,657 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:35:57,657 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,658 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 11:35:57,659 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-309096596] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,659 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 11:35:57,659 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#2085596096 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 11:35:57,659 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#2085596096] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,660 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:57,661 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:57,661 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 11:35:57,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:57,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:57,662 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:35:57,662 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 11:35:57,662 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 11:35:57,662 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:57,662 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 11:35:57,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@6d353e5c} 11:35:57,663 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 11:35:57,663 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:57,663 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:35:57,663 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 11:35:57,663 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,663 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:57,663 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 11:35:57,664 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:57,664 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#740008762] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,664 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#886863046] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,664 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 11:35:57,665 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:57,665 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 11:35:57,666 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#879311121 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 11:35:57,666 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:57,666 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#879311121] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,666 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-97009177 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 11:35:57,666 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-97009177] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:35:57,667 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:57,667 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 11:35:57,667 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:35:57,667 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 11:35:57,668 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:57,668 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:57,668 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:57,668 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 11:35:57,668 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 11:35:57,668 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 11:35:57,668 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:57,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:35:57,669 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:35:57,671 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:57,672 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:35:57,672 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 11:35:57,672 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:35:57,672 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:57,672 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:57,672 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 11:35:57,672 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:57,672 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 11:35:57,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:35:57,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 11:35:57,673 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:35:57,673 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:35:57,673 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 11:35:57,674 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 11:35:57,674 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 11:35:57,726 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 11:35:57,727 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 11:35:57,728 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 11:35:57,728 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 11:35:57,967 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 11:35:58,47 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:35:58,47 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:35:58,254 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:35:58,285 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:35:58,366 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:58,366 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:58,541 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5723284641946130884] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:35:58,545 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5723284641946130884] but current is [-3230057587588849994]. Reason: ActorSystem terminated 11:35:58,588 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 11:35:58,589 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:35:58,590 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 11:35:58,590 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 11:35:58,590 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 11:35:58,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1959586251]] intended for system uid [5723284641946130884], self uid is [-3230057587588849994] 11:35:58,597 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1628b562 11:35:58,598 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 11:35:58,598 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 11:35:58,598 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 11:35:58,599 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 11:35:58,599 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 11:35:58,599 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1aa6f83e 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b3df6e9 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25fa0dfe 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 11:35:58,599 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 11:35:58,599 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 11:35:58,599 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 11:35:58,600 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1186, lastApplied : -1, commitIndex : -1 11:35:58,600 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1186, lastApplied : -1, commitIndex : -1 11:35:58,600 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1140, lastApplied : 0, commitIndex : 0 11:35:58,600 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1267, lastApplied : -1, commitIndex : -1 11:35:58,600 AM [cluster-test-shard-dispatcher-20] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 11:35:58,601 AM [cluster-test-shard-dispatcher-20] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 11:35:58,601 AM [cluster-test-shard-dispatcher-20] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 11:35:58,601 AM [cluster-test-shard-dispatcher-20] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 11:35:58,606 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:35:58,609 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:35:58,609 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 11:35:58,688 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:58,688 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:35:58,924 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:35:58,926 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 11:35:58,927 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 326, lastApplied : -1, commitIndex : -1 11:35:59,6 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:35:59,7 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:35:59,7 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 406, lastApplied : -1, commitIndex : -1 11:35:59,8 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 407, lastApplied : -1, commitIndex : -1 11:35:59,8 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:59,8 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:59,35 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 426, lastApplied : 0, commitIndex : 0 11:35:59,246 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,246 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:35:59,326 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,327 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:35:59,327 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,328 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,355 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:35:59,566 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,566 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:59,624 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:35:59,646 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:35:59,646 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,647 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:35:59,647 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:59,675 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:35:59,887 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:59,887 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:35:59,967 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:35:59,968 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:35:59,969 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 11:35:59,970 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 11:35:59,996 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:00,206 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:00,207 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:00,285 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:00,285 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:36:00,286 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 11:36:00,286 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 11:36:00,316 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:00,525 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:00,526 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:00,607 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:00,607 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:00,608 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 11:36:00,608 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 11:36:00,636 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:00,644 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:00,846 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:00,847 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:00,927 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:00,927 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:00,929 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:00,930 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:00,956 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:01,166 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,167 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,246 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,246 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,247 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:36:01,247 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:36:01,276 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:01,486 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:01,487 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,567 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:01,567 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:01,567 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,567 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:01,595 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 11:36:01,664 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:01,806 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,806 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:01,885 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:01,885 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:01,885 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:01,885 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:01,916 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:02,126 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,127 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,206 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,206 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,207 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:02,207 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:02,236 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:02,443 AM [cluster-test-shard-dispatcher-23] [WARN] Leader - member-1-shard-people-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,443 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:02,443 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,443 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:02,443 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,446 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:02,446 AM [cluster-test-shard-dispatcher-23] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 11:36:02,447 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 11:36:02,447 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,447 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 11:36:02,447 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,447 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 11:36:02,447 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:36:02,523 AM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-people-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,523 AM [cluster-test-shard-dispatcher-23] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,524 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:02,524 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:02,524 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,524 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,524 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:02,524 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:02,524 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,524 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,527 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,527 AM [cluster-test-shard-dispatcher-20] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 11:36:02,527 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 11:36:02,527 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,527 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 11:36:02,528 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 11:36:02,528 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,528 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:36:02,528 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:02,528 AM [cluster-test-shard-dispatcher-23] [INFO] IsolatedLeader - member-1-shard-people-config_testRemoveShardReplica: IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 11:36:02,528 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 11:36:02,528 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,528 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 11:36:02,528 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 11:36:02,528 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:02,528 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 11:36:02,552 AM [cluster-test-shard-dispatcher-23] [WARN] Leader - member-1-shard-cars-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,553 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:02,553 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,553 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:02,553 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 11:36:02,555 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 11:36:02,555 AM [cluster-test-shard-dispatcher-23] [INFO] IsolatedLeader - member-1-shard-cars-config_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 11:36:02,555 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 11:36:02,555 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:02,555 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 11:36:02,555 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 11:36:02,555 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 11:36:02,562 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#905141734]] intended for system uid [5723284641946130884], self uid is [-3230057587588849994] 11:36:02,684 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:02,765 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 11:36:02,766 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 11:36:02,846 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 11:36:02,846 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 11:36:02,846 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 11:36:02,847 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 11:36:02,875 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:03,86 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,86 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:03,167 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,167 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,167 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,167 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,195 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:03,354 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 11:36:03,355 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:03,355 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:03,406 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,406 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,487 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,487 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,488 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,488 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:03,514 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:03,704 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:03,725 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,728 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:03,729 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:03,729 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:03,806 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,806 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,807 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,807 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:03,835 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:04,46 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:36:04,46 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:04,126 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:04,126 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:04,126 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,126 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,155 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:04,315 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1959586251]] intended for system uid [5723284641946130884], self uid is [-3230057587588849994] 11:36:04,365 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,366 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:04,446 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,447 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:04,447 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:04,447 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:04,476 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:04,651 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 11:36:04,685 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,685 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,725 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:04,766 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,766 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,766 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,766 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:04,795 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 11:36:05,6 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,7 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:05,85 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,86 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,86 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,86 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,115 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:05,326 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,328 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:05,406 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,406 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,406 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:05,407 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,435 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 11:36:05,647 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,648 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:05,673 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2559] 11:36:05,674 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:05,674 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:05,676 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5723284641946130884] but current is [-3230057587588849994]. Reason: Cluster member removed, previous status [Down] 11:36:05,726 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,726 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,727 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,727 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:05,744 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:05,755 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:05,764 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:05,764 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5723284641946130884] but current is [-3230057587588849994]. Reason: Cluster member removed, previous status [Down] 11:36:05,967 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:05,967 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,46 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,46 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,47 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:06,47 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:06,75 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 11:36:06,285 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:06,286 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:06,366 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,366 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,366 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,366 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:06,395 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:06,606 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:06,607 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:06,687 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:06,687 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:06,687 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:06,688 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:06,715 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 11:36:06,764 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:06,770 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:06,770 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:06,770 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 11:36:06,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:36:06,770 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:36:06,770 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:06,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:36:06,770 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 11:36:06,771 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 11:36:06,771 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 11:36:06,771 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1859027857] 11:36:06,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:06,773 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#879611252] 11:36:06,773 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#879611252] 11:36:06,773 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-2}, datastoreType Config 11:36:06,774 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 11:36:06,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:06,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2afa2dd3] 11:36:06,776 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028 for shard member-2-shard-cars-config_testRemoveShardReplica 11:36:06,777 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 11:36:06,777 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 11:36:06,777 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 11:36:06,777 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1403293535] 11:36:06,777 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:06,777 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:36:06,777 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#890587028] 11:36:06,778 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1859027857] 11:36:06,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:06,778 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#879611252] 11:36:06,778 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:06,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#879611252] with 27000 ms timeout 11:36:06,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 11:36:06,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 11:36:06,781 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 11:36:06,781 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 11:36:06,781 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#879611252] 11:36:06,828 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1859027857] 11:36:06,828 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:06,828 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 11:36:06,829 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 11:36:06,830 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 11:36:06,830 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 11:36:06,831 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 11:36:06,831 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 11:36:06,832 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:06,832 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:06,833 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 11:36:06,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 11:36:06,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 11:36:06,833 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:06,834 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 11:36:06,834 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 11:36:06,834 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 11:36:06,835 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 11:36:06,835 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:06,835 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:06,837 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:06,837 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:06,844 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:06,844 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:06,844 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:06,844 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:06,846 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:06,858 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:36:06,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 11:36:06,859 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 11:36:06,860 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Stopping Shard 11:36:06,861 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal closed 11:36:06,861 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:06,861 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:06,861 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 11:36:06,861 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 11:36:06,862 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 11:36:06,862 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:36:06,862 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Stopping Shard 11:36:06,862 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Stopping Shard 11:36:06,862 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal closed 11:36:06,862 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal closed 11:36:06,863 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:06,863 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:06,864 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:06,864 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:07,563 AM [cluster-test-shard-dispatcher-23] [WARN] Leader - member-1-shard-people-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:07,563 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:07,563 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:07,563 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:07,564 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:07,643 AM [cluster-test-shard-dispatcher-23] [WARN] Leader - member-1-shard-people-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 11:36:07,643 AM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:07,643 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:07,643 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:07,643 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:07,643 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:07,643 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:07,644 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 11:36:07,643 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 11:36:07,644 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 11:36:07,712 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:07,853 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-3230057587588849994] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:07,883 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:07,884 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [687338696277365272] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:07,885 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:07,886 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:07.888] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:07,892 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 11:36:07,893 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 11:36:07,893 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 11:36:07,893 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:36:07,893 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@56bc2af5 11:36:07,893 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@56bc2af5 11:36:07,894 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Stopping Shard 11:36:07,894 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Stopping Shard 11:36:07,894 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal closed 11:36:07,895 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal closed 11:36:07,895 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:07,895 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:07,896 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 11:36:07,896 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 11:36:07,896 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 11:36:07,896 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:07,897 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Stopping Shard 11:36:07,897 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Stopping Shard 11:36:07,897 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal closed 11:36:07,897 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal closed 11:36:07,897 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:07,897 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:07,899 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:07,899 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:07,901 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:07.903] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:07,926 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:07,935 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7421100538774644547] 11:36:07,936 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:07,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:07,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:07,943 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:07,943 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:07,951 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:07,951 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:07,951 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:07,951 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:07,953 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:07,955 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 11:36:07,961 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:07,961 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:07,961 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1797215023] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,964 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:07,965 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:07,965 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 11:36:07,966 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:07,966 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:07,966 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:07,969 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:07,970 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@579ed7aa} 11:36:07,970 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 11:36:07,971 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:07,973 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:07,974 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:07,974 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:07,975 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:07,975 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:07,975 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-642591816] 11:36:07,975 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 11:36:07,976 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:07,976 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:07,976 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#569369370] 11:36:07,976 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:07,976 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-642591816] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,977 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-991461563 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:07,977 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:07,977 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-991461563] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,979 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:07,979 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:07,980 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:07,981 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 11:36:07,981 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:07,982 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#569369370] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,982 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-672688087] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,982 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-672688087 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 11:36:07,983 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:07,983 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:07,983 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:07,983 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:07,984 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:07,984 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:07,985 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:07,985 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:07,985 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:07,985 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:07,985 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:07,985 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:07,987 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1631977589] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,990 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:07,991 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:07,991 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 11:36:07,991 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:07,991 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:07,991 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:07,991 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:07,991 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 11:36:07,991 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 11:36:07,991 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:07,992 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:07,992 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 11:36:07,992 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:07,992 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:07,992 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:07,993 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:07,993 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 11:36:07,993 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@691d7239} 11:36:07,993 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 11:36:07,994 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:07,994 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:07,996 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:07,996 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:07,996 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1190544834] 11:36:07,996 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:07,997 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:07,997 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:07,997 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#377409943] 11:36:07,998 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:07,998 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1190544834] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,998 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:07,998 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#377409943] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,998 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#1661637771 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:07,998 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#1661637771] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:07,998 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:07,999 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#970227314 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:07,999 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:07,999 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#970227314] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,0 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,0 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,2 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,3 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,3 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,3 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:08,6 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,7 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,7 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,7 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,7 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:08,7 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 11:36:08,9 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,10 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,10 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,10 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,10 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 11:36:08,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:08,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 11:36:08,10 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 11:36:08,11 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 11:36:08,11 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 11:36:08,21 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [3225086669036480972] 11:36:08,21 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:08,27 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:08,27 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:08,28 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:08,28 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:08,61 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,63 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,65 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 11:36:08,74 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:08,75 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:08,75 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1214789583] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,95 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:08,95 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,95 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,98 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:08,99 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@18d3f704} 11:36:08,99 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 11:36:08,100 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,102 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,102 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:08,102 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,103 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,104 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,104 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,105 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,106 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,106 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,107 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,107 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1545502350] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,107 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1871318545] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,107 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1175020672 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,107 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#-1098810234 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,107 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1175020672] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,108 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 11:36:08,115 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:08,115 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#-1098810234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,115 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1555914641] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,115 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:08,115 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,116 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,118 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:08,118 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,119 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,119 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,119 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,119 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,120 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,120 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,120 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:36:08,120 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 11:36:08,120 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:08,120 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 11:36:08,122 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,124 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 11:36:08,125 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:08,126 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@18aacc3d} 11:36:08,126 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 11:36:08,126 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:08,126 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,127 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,127 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,127 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:08,127 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 11:36:08,127 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,127 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,128 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,128 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,128 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:08,128 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#199830890] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,128 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#-1089642220] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,128 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,128 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-967537210 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,129 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-967537210] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,129 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-1195011531 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,129 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-2-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,129 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 11:36:08,130 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,130 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,131 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,131 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,133 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,134 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,134 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:08,134 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 11:36:08,134 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,134 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,134 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,135 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:08,135 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,135 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 11:36:08,135 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 11:36:08,135 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-1195011531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,137 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,138 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,140 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,140 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:08,142 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:36:08,142 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 11:36:08,142 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,143 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,143 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,143 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,143 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 11:36:08,143 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 11:36:08,158 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-123099030007498523] 11:36:08,159 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:08,162 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:08,162 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:08,165 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:08,166 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:08,174 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,176 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,178 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 11:36:08,190 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:08,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:08,191 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,205 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:08,207 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:08,209 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:08,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,210 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:08,210 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:08,210 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,210 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@17115c26} 11:36:08,211 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 11:36:08,212 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,214 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:08,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:08,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-3-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#165757386] 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-3-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1358731861] 11:36:08,216 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,216 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,216 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 11:36:08,216 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#165757386] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1358731861] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,217 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#-1230784981 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#821184100] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#-1230784981] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,218 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,218 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,219 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#821184100 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,219 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#938545281] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,220 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:08,220 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:08,221 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,221 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,221 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:08,221 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,221 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,222 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,222 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:08,222 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,222 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,222 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,227 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,228 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:08,228 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 11:36:08,228 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,228 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,228 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,228 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,229 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:08,229 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 11:36:08,229 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,230 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 11:36:08,230 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@26fa22f7} 11:36:08,230 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 11:36:08,230 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:36:08,230 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 11:36:08,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:08,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,231 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:08,232 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,232 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,232 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-212909056] 11:36:08,232 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,232 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-212909056] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,232 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,232 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,233 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,233 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1669391056] 11:36:08,233 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:08,233 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1669391056] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,233 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-544742784 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,234 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,234 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-544742784] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,234 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,234 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,234 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,234 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,234 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:08,234 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 11:36:08,235 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-731512389 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,235 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,235 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-731512389] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:08,236 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,236 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:08,236 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,238 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:08,239 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:08,240 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:08,240 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,241 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:36:08,241 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 11:36:08,241 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,241 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,241 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,241 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,241 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 11:36:08,241 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 11:36:08,244 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:08,244 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:08,244 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 11:36:08,244 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,244 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,244 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 11:36:08,244 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,245 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 11:36:08,245 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 11:36:08,357 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 11:36:08,358 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:08,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,358 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 11:36:08,358 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 11:36:08,367 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 11:36:08,367 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:08,367 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,367 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,367 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 11:36:08,368 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 11:36:08,368 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 11:36:08,368 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:08,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,368 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 11:36:08,368 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 11:36:08,397 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 11:36:08,397 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:08,397 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:08,397 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:08,397 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 11:36:08,398 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 11:36:08,689 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 11:36:08,747 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 11:36:08,757 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 11:36:08,807 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 11:36:08,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:08,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:08,956 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:08,956 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:08,956 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:08,956 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,956 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,957 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:08,965 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:09,38 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 11:36:09,41 AM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,42 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,55 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:09,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 11:36:09,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 11:36:09,55 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 11:36:09,55 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f80ce5e 11:36:09,55 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 11:36:09,56 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a467d57 11:36:09,57 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 11:36:09,58 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 11:36:09,58 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e5fa694 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:09,62 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:09,62 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:09,62 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:09,62 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:09,63 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 11:36:09,63 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:09,107 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 11:36:09,109 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,109 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,117 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 11:36:09,118 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:09,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 11:36:09,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 11:36:09,118 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 11:36:09,118 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c1fe17e 11:36:09,118 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 11:36:09,119 AM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,119 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,120 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e585a2a 11:36:09,120 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 11:36:09,120 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2481966f 11:36:09,120 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 11:36:09,127 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:09,128 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 11:36:09,128 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 11:36:09,128 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 11:36:09,128 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@114733de 11:36:09,128 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 11:36:09,128 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 11:36:09,128 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 11:36:09,128 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1797215023] 11:36:09,129 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11a27ca0 11:36:09,129 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:09,129 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-642591816] 11:36:09,129 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 11:36:09,129 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-642591816] 11:36:09,129 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 11:36:09,129 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1214789583] 11:36:09,130 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:09,130 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 11:36:09,130 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1545502350] 11:36:09,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1545502350] 11:36:09,130 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77755abe 11:36:09,130 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 11:36:09,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] 11:36:09,130 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:09,130 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#165757386] 11:36:09,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#165757386] 11:36:09,142 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@a132e9b] 11:36:09,142 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 11:36:09,144 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:09,145 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#2009114192 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 11:36:09,145 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-1-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:09,147 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:09,148 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:09,150 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:09,153 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:09,153 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 11:36:09,153 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 11:36:09,154 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:09,154 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:09,154 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:09,154 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@20be1627] 11:36:09,155 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 11:36:09,156 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:09,156 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:09,156 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#100508484 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 11:36:09,156 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:09,158 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:09,159 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 11:36:09,159 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:09,160 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:09,162 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:09,162 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 11:36:09,162 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 11:36:09,162 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:09,163 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:09,163 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:09,163 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:09,163 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 11:36:09,166 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@63fbd1f6] 11:36:09,167 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 11:36:09,168 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1797215023] 11:36:09,168 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 11:36:09,168 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 11:36:09,168 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#371780936] 11:36:09,168 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#371780936] 11:36:09,169 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1214789583] 11:36:09,169 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 11:36:09,169 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#719269142 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 11:36:09,169 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-3-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 11:36:09,169 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-306487509] 11:36:09,169 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-306487509] 11:36:09,169 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] 11:36:09,169 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 11:36:09,170 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 11:36:09,172 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:09,172 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:09,175 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 11:36:09,175 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:09,175 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:09,175 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 11:36:09,170 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:36:09,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 11:36:09,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 11:36:09,178 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 11:36:09,178 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1245836328] 11:36:09,178 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1245836328] 11:36:09,178 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 11:36:09,178 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 11:36:09,180 AM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 11:36:09,180 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 11:36:09,180 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 11:36:09,180 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 11:36:09,181 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 11:36:09,181 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 11:36:09,182 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 11:36:09,182 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 11:36:09,182 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 11:36:09,182 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 11:36:09,183 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:09,183 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 11:36:09,183 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:36:09,183 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:36:09,183 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 11:36:09,187 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 11:36:09,187 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 11:36:09,187 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:36:09,187 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 11:36:09,190 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,190 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 11:36:09,190 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:09,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 11:36:09,191 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:36:09,191 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 11:36:09,192 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 11:36:09,192 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 11:36:09,192 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:09,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 11:36:09,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 11:36:09,194 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,194 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:36:09,194 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,194 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 11:36:09,194 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1358731861] with 27000 ms timeout 11:36:09,194 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 11:36:09,194 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, cars] 11:36:09,194 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 11:36:09,195 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 11:36:09,195 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal closed 11:36:09,195 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1358731861] 11:36:09,197 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:09,198 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-212909056] with 27000 ms timeout 11:36:09,198 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 11:36:09,198 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [people] 11:36:09,198 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 11:36:09,198 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal closed 11:36:09,199 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-212909056] 11:36:09,201 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 11:36:09,202 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#165757386] with 27000 ms timeout 11:36:09,202 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 11:36:09,202 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 11:36:09,202 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 11:36:09,203 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal closed 11:36:09,203 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#165757386] 11:36:09,205 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:09,205 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 11:36:09,205 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 11:36:09,205 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 11:36:09,206 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 11:36:09,206 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6441bcf 11:36:09,206 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 11:36:09,206 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 11:36:09,207 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 11:36:09,207 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e0f1345 11:36:09,207 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 11:36:09,207 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 11:36:09,207 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21f53da4 11:36:09,207 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 11:36:09,207 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 11:36:09,208 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 11:36:09,210 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 11:36:09,210 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 11:36:09,211 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:36:09,211 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 11:36:09,214 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:09,214 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 11:36:09,215 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1669391056] with 27000 ms timeout 11:36:09,215 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 11:36:09,215 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 11:36:09,216 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 11:36:09,216 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal closed 11:36:09,217 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1669391056] 11:36:09,478 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 11:36:09,479 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:09,479 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 11:36:09,479 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:09,479 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 11:36:09,479 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 11:36:09,481 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,482 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:09,489 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:09,490 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 11:36:09,490 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 11:36:09,490 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 11:36:09,490 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@61ed5144 11:36:09,490 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 11:36:09,490 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 11:36:09,491 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 11:36:09,491 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b0d90e0 11:36:09,492 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 11:36:09,492 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@137a95c8 11:36:09,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 11:36:09,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 11:36:09,493 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 11:36:09,493 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 11:36:09,493 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 11:36:09,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 11:36:09,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 11:36:09,496 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 11:36:09,496 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 11:36:09,499 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1797215023] 11:36:09,499 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:09,499 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-642591816] 11:36:09,499 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-642591816] 11:36:09,499 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1797215023] 11:36:09,499 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:36:09,500 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#569369370] 11:36:09,500 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:09,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#569369370] 11:36:09,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1797215023] 11:36:09,500 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 11:36:09,500 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#371780936] 11:36:09,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#371780936] 11:36:09,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1214789583] 11:36:09,500 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:09,501 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 11:36:09,501 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1545502350] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1545502350] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1214789583] 11:36:09,501 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:36:09,501 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1871318545] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1871318545] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1214789583] 11:36:09,501 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 11:36:09,501 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-306487509] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-306487509] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] 11:36:09,501 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:09,501 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [pets] 11:36:09,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] 11:36:09,501 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:36:09,502 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [pets] 11:36:09,502 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] 11:36:09,502 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 11:36:09,502 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1245836328] 11:36:09,502 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1245836328] with 27000 ms timeout 11:36:09,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 11:36:09,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 11:36:09,503 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 11:36:09,503 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal closed 11:36:09,504 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1245836328] 11:36:09,526 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : 0, commitIndex : 0 11:36:09,552 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1725034727] 11:36:09,552 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 11:36:09,552 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 11:36:09,553 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:09,553 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 11:36:09,553 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:09,554 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:09,554 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 11:36:09,554 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 11:36:09,555 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:09,555 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:09,556 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:09,556 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:09,563 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:09,563 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:09,563 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:09,563 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:09,566 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:09,574 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:09,575 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 11:36:09,575 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 11:36:09,575 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 11:36:09,575 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 11:36:09,575 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 11:36:09,575 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 11:36:09,575 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 11:36:09,575 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal closed 11:36:09,575 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal closed 11:36:09,576 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal closed 11:36:09,576 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:09,576 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:09,577 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 11:36:09,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 11:36:09,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 11:36:09,577 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:09,578 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 11:36:09,578 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 11:36:09,578 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal closed 11:36:09,578 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal closed 11:36:09,579 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:09,579 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:09,579 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:09,579 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:09,975 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:10,574 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-123099030007498523] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:10,594 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [3225086669036480972] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:10,601 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:10,602 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:10,604 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:10.607] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:10,619 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 11:36:10,620 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 11:36:10,620 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 11:36:10,620 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 11:36:10,620 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 11:36:10,621 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ca0b79b 11:36:10,621 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ca0b79b 11:36:10,621 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 11:36:10,621 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1f7718b0 11:36:10,621 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@705e752f 11:36:10,621 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1f7718b0 11:36:10,621 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@705e752f 11:36:10,621 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 11:36:10,621 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 11:36:10,621 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@fe7e869 11:36:10,621 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e1dcc2d 11:36:10,621 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@23f2c636 11:36:10,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 11:36:10,974 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 353.8 ms 11:36:10,975 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 11:36:10,975 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 11:36:10,976 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal closed 11:36:11,44 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 11:36:11,44 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 423.5 ms 11:36:11,44 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 11:36:11,45 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 11:36:11,45 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal closed 11:36:11,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 11:36:11,84 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 463.5 ms 11:36:11,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 11:36:11,85 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 11:36:11,86 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal closed 11:36:11,86 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:11,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:11,87 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 11:36:11,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 11:36:11,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 11:36:11,88 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:11,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@757a2ff0 11:36:11,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@757a2ff0 11:36:11,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 11:36:11,88 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c301e31 11:36:11,88 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@75407cfe 11:36:11,88 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@75407cfe 11:36:11,88 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 11:36:11,88 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@772799f0 11:36:11,444 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 11:36:11,444 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 356.0 ms 11:36:11,444 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 11:36:11,445 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 11:36:11,446 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal closed 11:36:11,674 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 11:36:11,674 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 586.4 ms 11:36:11,675 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 11:36:11,677 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 11:36:11,677 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal closed 11:36:11,678 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:11,678 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:11,679 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:11,679 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:11,683 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:11.684] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:11,710 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:11,721 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4006046209874014549] 11:36:11,721 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:11,725 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:11,725 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:11,725 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:11,725 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:11,738 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:11,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:11,740 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:11,741 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,744 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 11:36:11,748 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:11,748 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:11,749 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#980246982] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,751 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:11,751 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,752 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:11,752 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 11:36:11,767 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@b4fcf3e} 11:36:11,767 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 11:36:11,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:11,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,768 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,769 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#1458640907] 11:36:11,769 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,770 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Shard created, persistent : true 11:36:11,770 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#1458640907] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,770 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1527801303 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,771 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1527801303] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,771 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplicaFailures: Starting recovery with journal batch size 1 11:36:11,771 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 11:36:11,773 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal open: applyTo=0 11:36:11,775 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-762216921] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,777 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:11,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:11,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:11,779 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:11,781 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:11,781 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,782 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:11,782 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,783 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 11:36:11,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:11,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 11:36:11,784 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,784 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,784 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 11:36:11,784 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 11:36:11,785 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@3a746c9c} 11:36:11,785 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 11:36:11,785 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:11,786 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,786 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 11:36:11,786 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,786 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,786 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-1383327412] 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 11:36:11,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 11:36:11,786 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 11:36:11,787 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 11:36:11,787 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@12a5c51a 11:36:11,787 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 11:36:11,787 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 11:36:11,787 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-1383327412] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,787 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Shard created, persistent : true 11:36:11,787 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-539426640 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,787 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=people} 11:36:11,787 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-539426640] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,787 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 11:36:11,788 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 11:36:11,788 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplicaFailures: Starting recovery with journal batch size 1 11:36:11,788 AM [cluster-test-pekko.actor.default-dispatcher-13] [ERROR] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=people} java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1162) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:246) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 11:36:11,788 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 11:36:11,789 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 11:36:11,789 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 11:36:11,789 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1c62293e 11:36:11,789 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1c62293e 11:36:11,789 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal open: applyTo=0 11:36:11,789 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Stopping Shard 11:36:11,790 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal closed 11:36:11,790 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:11,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:11,791 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:11,791 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:11,792 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 11:36:11,792 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 11:36:11,792 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#980246982] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#980246982] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,792 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#980246982] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#980246982] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,792 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 11:36:11,792 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f0a04803690] to Actor[pekko://cluster-test/user/$a#-1891930538] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-1891930538] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,794 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 11:36:11,794 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:11,794 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 11:36:11,794 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,794 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,794 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,797 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 11:36:11,797 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 11:36:11,797 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 11:36:11,797 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 11:36:11,797 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6d628149 11:36:11,797 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 11:36:11,797 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 11:36:11,806 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Stopping Shard 11:36:11,806 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal closed 11:36:11,806 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:11,807 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:11,808 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:11,808 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:11,809 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:11,830 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:11,845 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-3804812337142936638] 11:36:11,846 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:11,848 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:11,848 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:11,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:11,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:11,861 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:11,861 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,861 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:11,861 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:11,862 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,864 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 11:36:11,870 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-677316166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,870 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:11,870 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:11,876 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:11,876 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 11:36:11,876 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:11,876 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:11,878 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:11,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@57a6081f} 11:36:11,879 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 11:36:11,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:11,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 11:36:11,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:11,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:11,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:11,881 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Shard created, persistent : true 11:36:11,881 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,881 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#1666971064 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 11:36:11,881 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#1666971064] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,882 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 11:36:11,883 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal open: applyTo=0 11:36:11,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:11,886 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:11,888 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:11,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:11,889 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 11:36:11,889 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,889 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 11:36:11,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 11:36:11,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@30c69fe6 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 11:36:11,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 11:36:11,930 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-677316166] 11:36:11,930 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:11,930 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:11,930 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,933 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:11,936 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 11:36:11,938 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-777493175] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,939 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:11,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:11,942 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:11,942 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 11:36:11,942 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:11,943 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 11:36:11,944 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:11,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4a047d0d} 11:36:11,945 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 11:36:11,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:11,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 11:36:11,946 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 11:36:11,946 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 11:36:11,946 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] 11:36:11,946 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 11:36:11,946 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,946 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#274129694 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 11:36:11,946 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 11:36:11,947 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#274129694] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:11,947 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 11:36:11,948 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:11,948 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:11,951 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:11,951 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:11,951 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 11:36:11,952 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,952 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,952 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 11:36:11,954 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 11:36:11,954 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:11,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:11,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:11,954 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 11:36:11,954 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:11,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:11,955 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 11:36:11,955 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 11:36:11,955 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 11:36:11,955 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68e682df 11:36:11,955 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 11:36:11,955 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:11,995 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-777493175] 11:36:11,995 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:11,995 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] 11:36:12,4 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:12,14 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1823389065707007637] 11:36:12,14 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:12,18 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:12,18 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:12,18 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:12,18 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:12,30 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:12,32 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:12,35 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 11:36:12,41 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,41 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:12,42 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:12,48 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:12,53 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:12,54 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 11:36:12,54 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:12,54 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@33005c1c} 11:36:12,55 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:12,55 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:12,58 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 11:36:12,60 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-829492630] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,61 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:12,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:12,63 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:12,64 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 11:36:12,65 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:12,65 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@3dc7b40c} 11:36:12,65 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:12,70 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:12,72 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:12,74 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:12,74 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 11:36:12,74 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:12,74 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:12,865 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:12,866 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:12,866 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 11:36:12,866 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 11:36:12,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:12,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 11:36:12,867 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:12,867 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:36:12,867 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:12,867 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 11:36:12,867 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 11:36:12,868 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:12,868 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 11:36:12,870 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:36:12,870 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234, primaryVersion=13] 11:36:12,873 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234 for shard member-2-shard-cars-config_testAddShardReplica 11:36:12,874 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Shard created, persistent : true 11:36:12,874 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,874 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 11:36:12,874 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-431018813 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 11:36:12,875 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-431018813] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,876 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal open: applyTo=0 11:36:12,876 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 11:36:12,877 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:12,877 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:12,878 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:36:12,878 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:36:12,881 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:12,882 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:12,882 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 11:36:12,882 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:12,882 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:12,882 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 11:36:12,882 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:36:12,883 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:12.878273958Z 11:36:12,886 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 11:36:12,886 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:36:12,886 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 11:36:12,886 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 11:36:12,886 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5cb907ef 11:36:12,886 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 11:36:12,886 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 11:36:12,889 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:12.886778554Z 11:36:12,889 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 11:36:12,890 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:36:12,890 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-1645686925] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 11:36:12,890 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-1645686925] 11:36:12,890 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 399.0 μs 11:36:12,890 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 11:36:12,890 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 11:36:12,890 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 162.2 μs 11:36:12,891 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-1645686925] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,891 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:36:12,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234 11:36:12,894 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 11:36:12,894 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 11:36:12,894 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 11:36:12,898 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:12,899 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:36:12,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] 11:36:12,899 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@72bae90 11:36:12,899 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 11:36:12,899 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:12,899 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] 11:36:12,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 11:36:12,900 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] 11:36:12,900 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:12,900 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-829492630] 11:36:12,900 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:12,900 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 11:36:12,900 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 11:36:12,901 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:36:12,901 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 11:36:12,901 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:12,901 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 11:36:12,903 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:36:12,903 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360, primaryVersion=13] 11:36:12,905 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360 for shard member-2-shard-cars-oper_testAddShardReplica 11:36:12,906 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 11:36:12,906 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,907 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-320825343 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 11:36:12,907 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-320825343] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,907 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 11:36:12,908 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 11:36:12,908 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:36:12,908 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 11:36:12,911 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 11:36:12,911 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:12.908136480Z 11:36:12,912 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:12,913 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:12,915 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:12,916 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:12,916 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 11:36:12,916 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:12,916 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:12,916 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 11:36:12,916 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 11:36:12,920 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 11:36:12,920 AM [cluster-test-shard-dispatcher-26] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:36:12,920 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 11:36:12,920 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 11:36:12,920 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@269ac659 11:36:12,920 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 11:36:12,920 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:12,923 AM [cluster-test-shard-dispatcher-26] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:12.920509747Z 11:36:12,923 AM [cluster-test-shard-dispatcher-26] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 11:36:12,923 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:36:12,923 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-659739635] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 11:36:12,924 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-659739635] 11:36:12,924 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 572.8 μs 11:36:12,924 AM [cluster-test-shard-dispatcher-26] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 11:36:12,924 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 11:36:12,924 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-659739635] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,924 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 119.0 μs 11:36:12,925 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 11:36:12,927 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360 11:36:12,927 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 11:36:12,929 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 11:36:12,929 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 11:36:12,931 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:12,932 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4aef7acc 11:36:12,932 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:12,932 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 11:36:12,933 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 11:36:12,933 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:36:12,933 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:12,933 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-829492630] 11:36:12,933 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:12,933 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] 11:36:12,933 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] 11:36:12,939 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:12,957 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [3054629942477722508] 11:36:12,957 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:12,959 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:12,959 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:12,962 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:12,962 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:12,977 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:12,980 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:12,982 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 11:36:12,999 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:12,999 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:12,999 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:13,2 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:13,6 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:13,7 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@8b9b027} 11:36:13,7 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 11:36:13,8 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:13,8 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:13,13 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:13,22 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 11:36:13,30 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-488388485] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:13,34 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:13,35 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:13,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:13,36 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:13,36 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 11:36:13,36 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 11:36:13,36 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 11:36:13,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:13,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 11:36:13,37 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 11:36:13,37 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:13,38 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:13,38 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:13,38 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@7c65e902} 11:36:13,39 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 11:36:13,39 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:13,199 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 11:36:13,516 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 11:36:13,547 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 11:36:13,866 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 11:36:13,887 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:13,888 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:13,888 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:13,888 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 11:36:13,888 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 11:36:13,888 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:13,888 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 11:36:13,888 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 11:36:13,888 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:13,894 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:13,894 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:13,894 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:13,894 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:13,978 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:36:13,983 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:13,983 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 11:36:13,983 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 11:36:13,983 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 11:36:13,983 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:13,983 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 11:36:13,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 11:36:13,983 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:14,42 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:36:14,42 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 11:36:14,44 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,44 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 11:36:14,47 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,47 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234, primaryVersion=13] 11:36:14,53 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234 for shard member-3-shard-cars-config_testAddShardReplica 11:36:14,54 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 11:36:14,54 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#70636166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,55 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#278707968] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,55 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#278707968 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 11:36:14,55 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 11:36:14,57 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 11:36:14,57 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=0 11:36:14,58 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 11:36:14,58 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:36:14,60 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:14,61 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:14,65 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:14.058703061Z 11:36:14,65 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:14,65 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:14,65 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 11:36:14,65 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:14,66 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:14,66 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 11:36:14,66 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 11:36:14,68 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 11:36:14,68 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:36:14,68 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 11:36:14,68 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 11:36:14,69 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@67e4f40 11:36:14,69 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 11:36:14,69 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:14,71 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:14.069231448Z 11:36:14,71 AM [cluster-test-shard-dispatcher-15] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 11:36:14,71 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:36:14,72 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1325566124] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 11:36:14,72 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1325566124] 11:36:14,72 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 532.7 μs 11:36:14,72 AM [cluster-test-shard-dispatcher-15] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 11:36:14,72 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 11:36:14,72 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1325566124] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,72 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 101.6 μs 11:36:14,74 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 11:36:14,76 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234 11:36:14,76 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 11:36:14,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 11:36:14,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 11:36:14,78 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:14,81 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 11:36:14,81 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:14,81 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@241db57 11:36:14,81 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:36:14,81 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:14,81 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 11:36:14,81 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] 11:36:14,81 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:14,81 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#70636166] 11:36:14,81 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#70636166] 11:36:14,82 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-488388485] 11:36:14,82 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:14,82 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 11:36:14,82 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 11:36:14,82 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 11:36:14,82 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,82 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 11:36:14,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360, primaryVersion=13] 11:36:14,86 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360 for shard member-3-shard-cars-oper_testAddShardReplica 11:36:14,87 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 11:36:14,87 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 11:36:14,87 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1265174052] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,88 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#2125165855 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 11:36:14,88 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 11:36:14,88 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#2125165855] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,88 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 11:36:14,88 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-3-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 11:36:14,90 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 11:36:14,91 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:14.088223260Z 11:36:14,91 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:14,91 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:14,94 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:14,94 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:14,94 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:14,94 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 11:36:14,94 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:14,94 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 11:36:14,94 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 11:36:14,98 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 11:36:14,98 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 11:36:14,98 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 11:36:14,98 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 11:36:14,98 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4743457b 11:36:14,98 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 11:36:14,98 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:14,101 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-21T11:36:14.098756798Z 11:36:14,101 AM [cluster-test-shard-dispatcher-15] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 11:36:14,101 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:36:14,102 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-71700263] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 11:36:14,102 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 363.1 μs 11:36:14,102 AM [cluster-test-shard-dispatcher-15] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 11:36:14,102 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-71700263] 11:36:14,102 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 11:36:14,102 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-71700263] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,102 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 148.9 μs 11:36:14,103 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 11:36:14,107 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360 11:36:14,107 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 11:36:14,107 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 11:36:14,107 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 11:36:14,107 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:14,111 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2beed3c 11:36:14,111 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 11:36:14,111 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:14,112 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-488388485] 11:36:14,112 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:14,112 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1265174052] 11:36:14,112 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1265174052] 11:36:14,113 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] 11:36:14,113 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 11:36:14,113 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:14,113 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 11:36:14,113 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:14,113 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] 11:36:14,113 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] 11:36:14,113 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-829492630] 11:36:14,114 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:14,114 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] 11:36:14,114 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] 11:36:14,157 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] 11:36:14,158 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,158 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 11:36:14,158 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 11:36:14,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 11:36:14,181 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=76472, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:14,189 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:36:14,191 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:36:14,216 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 26.01 ms 11:36:14,221 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,241 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,241 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 11:36:14,245 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 11:36:14,245 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 11:36:14,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 11:36:14,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:36:14,246 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 11:36:14,247 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,250 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 11:36:14,251 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:36:14,256 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 11:36:14,256 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:36:14,256 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 11:36:14,256 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:36:14,256 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 11:36:14,257 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,263 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 11:36:14,265 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 11:36:14,271 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 11:36:14,273 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,275 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@70bde3a4}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,276 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,283 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,283 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:36:14,283 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:36:14,285 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#748400027] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 11:36:14,285 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#748400027] 11:36:14,285 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 11:36:14,285 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 129.8 μs 11:36:14,286 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#748400027] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] 11:36:14,300 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,301 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@61243ae8}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,302 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,302 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,302 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 11:36:14,302 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 140.0 μs 11:36:14,302 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,302 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 11:36:14,302 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 11:36:14,302 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 11:36:14,302 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,303 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@119dc17f}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,303 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,303 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,303 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 11:36:14,303 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 192.0 μs 11:36:14,306 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,312 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 11:36:14,312 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=10900, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:14,313 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:36:14,313 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:36:14,314 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 594.2 μs 11:36:14,318 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,326 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-488388485] 11:36:14,326 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,326 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 11:36:14,326 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 11:36:14,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 11:36:14,327 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=16850, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:14,328 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:36:14,328 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:36:14,329 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 767.6 μs 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 11:36:14,331 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,332 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 11:36:14,332 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 11:36:14,334 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,334 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@44938337}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,335 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,335 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,335 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:36:14,335 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:36:14,335 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 11:36:14,335 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#414571577] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 11:36:14,335 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#414571577] 11:36:14,335 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 11:36:14,335 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 74.74 μs 11:36:14,335 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#414571577] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,336 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-829492630] 11:36:14,336 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,336 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 11:36:14,336 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 11:36:14,337 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,338 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 11:36:14,338 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=12460, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:14,338 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,339 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:36:14,339 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:36:14,339 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@c438804}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,339 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,339 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,339 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 11:36:14,339 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 263.7 μs 11:36:14,339 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 153.1 μs 11:36:14,340 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,340 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a8c0690}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,340 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,340 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,340 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 11:36:14,340 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 100.8 μs 11:36:14,341 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 11:36:14,341 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,342 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f0a0496e358@7c1d3d1f 11:36:14,343 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-677316166] 11:36:14,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:14,343 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,343 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,347 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,347 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,348 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,348 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,394 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,445 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,496 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,546 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,597 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,647 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,698 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,799 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f0a0496e358@10fa2b91 11:36:14,799 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] 11:36:14,800 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,800 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,800 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,801 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-777493175] 11:36:14,802 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:14,802 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] 11:36:14,802 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] 11:36:14,802 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,802 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,802 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,802 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,803 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] 11:36:14,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:14,803 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] 11:36:14,803 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1115334559] 11:36:14,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-829492630] 11:36:14,804 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:14,804 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] 11:36:14,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#2008104413] 11:36:14,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] 11:36:14,804 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:14,804 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#70636166] 11:36:14,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#70636166] 11:36:14,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-488388485] 11:36:14,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:14,805 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1265174052] 11:36:14,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#1265174052] 11:36:14,805 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:14,806 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 11:36:14,806 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 11:36:14,806 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 11:36:14,807 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 11:36:14,808 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 11:36:14,808 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:14,808 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:14,808 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:14,808 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 11:36:14,808 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 11:36:14,809 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,809 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Stopping Shard 11:36:14,809 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1986466741] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,809 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal closed 11:36:14,809 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:14,809 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:14,810 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 11:36:14,810 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 11:36:14,812 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 11:36:14,812 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 11:36:14,813 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 11:36:14,813 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:14,813 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:14,825 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 11:36:14,826 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 11:36:14,827 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:14,827 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:14,827 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:14,827 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:14,829 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: aborting transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,832 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:14,834 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda/0x00007f0a049768f0@f62cac7 11:36:14,835 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@38887940}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,835 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,835 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,838 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:14,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:14,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-1761553966086052350] 11:36:14,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:14,872 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:14,872 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:14,876 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:14,876 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:14,885 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:14,887 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:14,891 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 11:36:14,897 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:14,899 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1795641695] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,898 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: applying snapshot ShardManagerSnapshot [ShardList = [cars] ] 11:36:14,899 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 11:36:14,904 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:14,904 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 11:36:14,907 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5a4dea0a} 11:36:14,908 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 11:36:14,909 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:14,909 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 11:36:14,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1795641695] 11:36:14,910 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 11:36:14,910 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1294081459] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,911 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1449238916 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 11:36:14,911 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1449238916] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,911 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 11:36:14,911 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:14,911 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 11:36:14,926 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=3 11:36:14,927 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:14,929 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 11:36:14,935 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:36:14,938 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-358979248] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 11:36:14,938 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-358979248] 11:36:14,938 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 2.391 ms 11:36:14,938 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 8.940 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 11:36:14,938 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 9.096 ms - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 0 11:36:14,938 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-358979248] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:14,938 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 11:36:14,939 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 196.4 μs 11:36:14,939 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 11:36:14,944 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@51897ca8}, purged=MutableUnsignedLongSet{size=0}} 11:36:14,945 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,945 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,945 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 11:36:14,945 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:14,945 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 11:36:14,945 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:14,946 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 11:36:14,946 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 11:36:14,946 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:14,946 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:14,946 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1294081459] 11:36:14,946 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 11:36:14,946 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 11:36:14,946 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1294081459] 11:36:14,946 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 11:36:14,946 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1795641695] 11:36:14,947 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:14,947 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 11:36:15,146 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5c426a93}, purged=MutableUnsignedLongSet{size=0}} 11:36:15,146 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:15,146 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:15,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f0a0496e358@7d78119b 11:36:15,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:15,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:15,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:15,149 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 11:36:15,149 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 11:36:15,824 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [3054629942477722508] but current is [-1761553966086052350]. Reason: ActorSystem terminated 11:36:15,831 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3054629942477722508] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:15,867 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] LocalActorRefProvider$$anon$2 - pekkoDeadLetter Message [org.apache.pekko.cluster.GossipStatus] wrapped in [org.apache.pekko.actor.ActorSelectionMessage] to Actor[pekko://cluster-test/] was dropped. Unknown system with UID [-1823389065707007637]. This system with UID [-1761553966086052350] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:15,867 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:15,893 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:15,953 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:15,953 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:15,953 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:15,953 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 11:36:15,954 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 11:36:15,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@460c3ec4 11:36:15,954 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 11:36:16,107 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:16,397 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:16,427 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:16,913 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:17,934 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:18,953 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:19,937 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1240799193]] intended for system uid [3054629942477722508], self uid is [-1761553966086052350] 11:36:19,973 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:20,7 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:20,7 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:20,7 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 11:36:20,11 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:20,11 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:20,103 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-1433456624]] intended for system uid [3054629942477722508], self uid is [-1761553966086052350] 11:36:20,172 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 11:36:20,993 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:36:21,26 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 11:36:21,27 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [3054629942477722508] but current is [-1761553966086052350]. Reason: Cluster member removed, previous status [Down] 11:36:21,27 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:21,27 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:21,199 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:21,199 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:21,200 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [3054629942477722508] but current is [-1761553966086052350]. Reason: Cluster member removed, previous status [Down] 11:36:22,14 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:22,19 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 11:36:22,19 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 11:36:22,19 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 11:36:22,20 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 11:36:22,20 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 11:36:22,20 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 11:36:22,22 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=7704217892, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 11:36:22,25 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=16941, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 11:36:22,26 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:36:22,26 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:36:22,26 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 481.0 μs 11:36:22,28 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 11:36:22,30 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 11:36:22,31 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 11:36:22,31 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 11:36:22,31 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 11:36:22,32 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 11:36:22,32 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:22,32 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:22,34 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:22,34 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:22,41 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:22,41 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:22,41 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:22,41 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:22,43 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:22,53 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:22,54 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 11:36:22,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 11:36:22,54 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Stopping Shard 11:36:22,54 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal closed 11:36:22,55 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:22,56 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:22,57 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:22,57 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f0a04803690] to Actor[pekko://cluster-test/user/$a#-820543338] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-820543338] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:22,57 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1785481549] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:22,57 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 11:36:22,57 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 11:36:22,57 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Stopping Shard 11:36:22,58 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal closed 11:36:22,58 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:22,58 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:22,59 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:22,59 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:23,55 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-1761553966086052350] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:23,65 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:23,66 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:23,66 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 11:36:23,66 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:23,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 11:36:23,67 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 11:36:23,67 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 11:36:23,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 11:36:23,67 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-1-shard-cars-oper_testAddShardReplica: Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 11:36:23,67 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 11:36:23,67 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testAddShardReplica: Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 11:36:23,73 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:23,74 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:23,75 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1823389065707007637] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:23,76 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:23.079] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:23,82 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 11:36:23,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 11:36:23,83 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 11:36:23,83 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42904b85 11:36:23,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f8291eb 11:36:23,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f8291eb 11:36:23,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 11:36:24,85 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:27,146 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 11:36:27,146 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:27,146 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:29,187 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:29,187 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:29,188 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 11:36:29,188 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:29,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:36:29,255 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1753963234] to Actor[pekko://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:29,255 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#170106360] to Actor[pekko://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:30,461 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:36:32,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 11:36:32,375 AM [cluster-test-shard-dispatcher-27] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.292 s 11:36:32,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 11:36:32,378 AM [cluster-test-shard-dispatcher-27] [WARN] Leader - member-1-shard-cars-config_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 11:36:32,378 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:36:32,378 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:32,378 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Leader to IsolatedLeader 11:36:32,378 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 11:36:32,379 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Stopping Shard 11:36:32,379 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal closed 11:36:32,380 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:32,380 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:32,380 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 11:36:32,381 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 11:36:32,381 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 11:36:32,381 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@568a62f4 11:36:32,381 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1d12e4dc 11:36:32,381 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1d12e4dc 11:36:32,381 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 11:36:32,415 AM [cluster-test-shard-dispatcher-12] [WARN] Leader - member-1-shard-cars-oper_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 11:36:32,415 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 11:36:32,415 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 11:36:32,415 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 34.25 ms 11:36:32,415 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 11:36:32,416 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:32,416 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 11:36:32,416 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 11:36:32,416 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Stopping Shard 11:36:32,417 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal closed 11:36:32,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:32,417 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:32,419 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:32,419 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:32,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:32.425] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:32,450 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:32,462 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7071207415276253332] 11:36:32,462 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:32,466 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:32,466 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:32,467 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:32,467 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:32,479 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,480 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:32,480 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:32,480 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:32,483 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,490 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 11:36:32,491 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,497 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-2054662523] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,497 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:32,498 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:32,498 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:32,498 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,499 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,499 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:32,499 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,499 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,504 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,505 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@27fb5b74} 11:36:32,505 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 11:36:32,506 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,508 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,510 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,510 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:32,511 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#261018843] 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,513 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1620464025] 11:36:32,514 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:32,514 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:32,514 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,514 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1631385742] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,514 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1620464025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,514 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#550767478 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,515 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#261018843] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,515 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#550767478] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,515 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,515 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:32,516 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,516 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,517 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,517 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,518 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-947476379 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,518 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-947476379] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,519 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,519 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,519 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,520 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 11:36:32,520 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,522 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,522 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,522 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,522 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,522 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:32,522 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,523 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,523 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,523 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,523 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,524 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:32,524 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,524 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,524 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@25f53549} 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:32,526 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#449033408] 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,526 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,526 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-792889545] 11:36:32,527 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,527 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,527 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,527 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#449033408] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,527 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-792889545] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,527 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,528 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#259339280 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,528 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#259339280] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,528 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1893521710 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,528 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1893521710] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,529 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,529 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,529 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,529 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:32,529 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,529 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,530 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,530 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,530 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,530 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,530 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,531 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,531 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,531 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:32,534 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,534 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,534 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,535 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:32,535 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,535 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,535 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,535 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,535 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,536 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:32,536 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,536 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,536 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,536 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,539 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7843985060207025895] 11:36:32,540 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:32,540 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,540 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,542 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:32,542 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:32,543 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:32,543 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:32,557 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,571 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,575 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,578 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1387348330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,583 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:32,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:32,587 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:32,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,590 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,591 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:32,593 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@3c45a4fd} 11:36:32,593 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 11:36:32,593 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,596 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:32,596 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,598 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:32,598 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,599 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,600 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:32,600 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,600 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,600 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,600 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,600 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,600 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,601 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,601 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,602 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-373240869 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,602 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1661188081 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,602 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1909789603] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,602 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,602 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,603 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,603 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,603 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,603 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,604 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:32,604 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:32,605 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:32,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,605 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,606 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:32,606 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,606 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,606 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#109118248] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,606 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-373240869] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,606 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1661188081] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,606 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,606 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#284979994] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,606 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,607 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:36:32,607 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,607 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:32,607 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,607 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,607 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,607 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,607 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,607 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,607 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,607 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,607 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,607 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,607 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,609 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,609 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@dae05b5} 11:36:32,609 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 11:36:32,610 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:32,610 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,610 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,610 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,610 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1052042962] 11:36:32,610 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,610 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,611 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,611 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,611 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1033924016] 11:36:32,611 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1052042962] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,611 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1033924016] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,611 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1639410737 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,611 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,611 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1639410737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,611 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,611 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-577613118 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,611 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,612 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,612 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-577613118] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,613 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,613 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,613 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,613 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,613 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,616 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,616 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,616 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:36:32,616 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,616 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:32,616 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,616 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,616 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,617 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,617 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,617 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,617 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,617 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,617 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,617 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,617 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,617 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,617 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,623 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:32,633 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [3639196186729014771] 11:36:32,634 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:32,642 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:32,642 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:32,642 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:32,642 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:32,649 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,651 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,653 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,673 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#218489221] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,675 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:32,676 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:32,681 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:32,681 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,681 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,682 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,683 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4a6ab424} 11:36:32,683 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 11:36:32,684 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,685 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:32,686 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:32,686 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:32,686 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,689 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:32,691 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,692 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,693 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,693 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1291051843] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,693 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1826726001 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1826726001] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,694 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:32,695 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,695 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,695 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,695 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,695 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:32,695 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:32,696 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,696 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#255024451 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,697 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,698 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,698 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1222487565] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,704 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,705 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:32,705 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,705 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,705 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,706 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,706 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,707 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,707 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-359883738] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,707 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#255024451] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,707 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,707 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:32,708 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,708 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,709 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,709 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,709 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,709 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,709 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:32,710 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,710 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:36:32,710 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,710 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,710 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,710 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,710 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:32,710 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,710 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:32,710 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,710 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,710 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 11:36:32,710 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@796572e4} 11:36:32,711 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 11:36:32,711 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:32,711 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,711 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:32,710 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,711 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,711 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,711 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,711 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-649998719] 11:36:32,711 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,712 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,712 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,712 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-649998719] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,712 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,712 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#444521403] 11:36:32,712 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1393520943 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,712 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,712 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 11:36:32,712 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#444521403] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,712 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1393520943] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,713 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 11:36:32,713 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,714 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#614511722 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,714 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#614511722] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:32,714 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,714 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,715 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 11:36:32,716 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:32,716 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:32,716 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,717 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,717 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,717 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:32,717 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:32,717 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,717 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,719 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 11:36:32,719 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:36:32,719 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 11:36:32,719 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,719 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,719 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 11:36:32,719 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:32,719 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 11:36:32,719 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 11:36:32,875 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 11:36:32,875 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:32,875 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,875 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,876 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 11:36:32,876 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 11:36:32,876 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 11:36:32,876 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:32,876 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,876 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,876 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 11:36:32,877 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 11:36:32,911 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 11:36:32,911 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:32,911 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,911 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,911 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 11:36:32,911 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 11:36:32,931 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 11:36:32,931 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:32,931 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 11:36:32,931 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:32,931 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 11:36:32,931 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 11:36:33,222 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 11:36:33,252 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 11:36:33,320 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 11:36:33,350 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 11:36:33,478 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:33,480 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:33,481 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:33,481 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:33,481 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:33,481 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,481 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,482 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,482 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,483 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:33,483 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,483 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,483 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,483 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,487 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,488 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,578 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,600 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 11:36:33,602 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,602 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,610 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:33,610 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:36:33,610 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 11:36:33,610 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 11:36:33,610 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11cc88d 11:36:33,610 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 11:36:33,612 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,612 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@16875161 11:36:33,612 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d06fb1d 11:36:33,612 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,641 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 11:36:33,642 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,642 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,650 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:33,650 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:36:33,650 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 11:36:33,650 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 11:36:33,650 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@441e46ca 11:36:33,650 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 11:36:33,651 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1eeb9119 11:36:33,651 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,651 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c22f771 11:36:33,651 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:36:33,660 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 11:36:33,661 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,661 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,668 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:33,668 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:36:33,668 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 11:36:33,668 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 11:36:33,669 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a4bfeff 11:36:33,669 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 11:36:33,669 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 11:36:33,669 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 11:36:33,669 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,669 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,669 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,669 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a3f247 11:36:33,669 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 11:36:33,669 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@134ec0cd 11:36:33,670 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 11:36:33,700 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 11:36:33,702 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,703 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:33,710 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:33,710 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 11:36:33,710 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 11:36:33,710 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 11:36:33,710 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75106383 11:36:33,711 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 11:36:33,711 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 11:36:33,711 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 11:36:33,711 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,711 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 11:36:33,711 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,711 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,711 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@79dddccb 11:36:33,711 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33fe59d5 11:36:33,711 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 11:36:33,711 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 11:36:33,711 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 11:36:33,711 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 11:36:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-2054662523] 11:36:33,711 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:33,712 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#261018843] 11:36:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#261018843] 11:36:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1387348330] 11:36:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:33,712 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1909789603] 11:36:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1909789603] 11:36:33,713 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#218489221] 11:36:33,713 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:33,713 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1291051843] 11:36:33,713 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1291051843] 11:36:33,720 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 11:36:33,720 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 11:36:33,721 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 11:36:33,721 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 11:36:33,721 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 11:36:33,722 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 11:36:33,722 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 11:36:33,723 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:33,723 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:36:33,723 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:33,723 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:36:33,723 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,723 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,723 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,723 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,726 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:33,726 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:33,726 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:33,727 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:36:33,729 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:36:33,730 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 11:36:33,731 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-2054662523] 11:36:33,731 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:33,731 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#261018843] 11:36:33,731 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#261018843] 11:36:33,732 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-2054662523] 11:36:33,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:36:33,732 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1620464025] 11:36:33,732 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1620464025] 11:36:33,732 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1631385742] 11:36:33,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:33,732 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#449033408] 11:36:33,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#449033408] 11:36:33,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1631385742] 11:36:33,733 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:36:33,733 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-792889545] 11:36:33,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-792889545] 11:36:33,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1387348330] 11:36:33,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:33,734 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1909789603] 11:36:33,734 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1909789603] 11:36:33,736 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1387348330] 11:36:33,739 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:36:33,739 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,739 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#109118248] 11:36:33,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#109118248] 11:36:33,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#284979994] 11:36:33,740 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:33,740 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1052042962] 11:36:33,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1052042962] 11:36:33,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#284979994] 11:36:33,740 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 11:36:33,740 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1033924016] 11:36:33,740 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 11:36:33,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1033924016] 11:36:33,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#218489221] 11:36:33,740 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:33,740 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1291051843] 11:36:33,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1291051843] 11:36:33,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#218489221] 11:36:33,741 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:36:33,741 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-359883738] 11:36:33,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-359883738] 11:36:33,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1222487565] 11:36:33,741 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:33,741 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-649998719] 11:36:33,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-649998719] 11:36:33,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1222487565] 11:36:33,741 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:36:33,741 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#444521403] 11:36:33,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#444521403] 11:36:33,742 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,742 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,742 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 11:36:33,743 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,743 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,743 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,743 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,744 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:33,744 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:33,744 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,744 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:36:33,745 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,745 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,745 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,745 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,745 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:33,745 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:33,746 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:33,746 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:33,750 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:33,750 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:33,750 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:33,751 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:33,753 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:33,766 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,767 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:33,767 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:33,767 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:36:33,767 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,768 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,768 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,768 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,769 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:33,769 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:33,769 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:33,769 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:33,770 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:33,770 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:33,770 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,770 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:33,770 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,770 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 11:36:33,771 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:33,771 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:33,772 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:33,773 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:34,499 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:34,768 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3639196186729014771] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:34,786 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:34,787 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:34,788 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7843985060207025895] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:34,789 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:34.791] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:34,796 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:34,797 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 11:36:34,797 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 11:36:34,797 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:36:34,797 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@380e4b76 11:36:34,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56cccdbe 11:36:34,797 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@67426225 11:36:34,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56cccdbe 11:36:34,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 11:36:34,797 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@67426225 11:36:34,797 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 11:36:34,797 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@16c04803 11:36:35,247 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 11:36:35,248 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 451.0 ms 11:36:35,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 11:36:35,250 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:35,252 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 11:36:35,267 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 11:36:35,268 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 470.5 ms 11:36:35,268 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 11:36:35,268 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:35,268 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 11:36:35,269 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:35,269 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:35,270 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 11:36:35,271 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 11:36:35,271 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 11:36:35,271 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:35,272 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33e93f92 11:36:35,272 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9ab0c24 11:36:35,272 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9ab0c24 11:36:35,272 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 11:36:35,272 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ddc3d8f 11:36:35,272 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ddc3d8f 11:36:35,272 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 11:36:35,272 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@628f375d 11:36:35,627 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 11:36:35,628 AM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 356.2 ms 11:36:35,628 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 11:36:35,628 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:35,629 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 11:36:35,847 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 11:36:35,847 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 575.9 ms 11:36:35,847 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 11:36:35,848 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 11:36:35,849 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 11:36:35,849 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:35,849 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:35,851 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:35,851 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:35,855 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:35.858] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:35,885 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:35,893 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-5444689083113798562] 11:36:35,894 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:35,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:35,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:35,900 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:35,900 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:35,912 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:35,913 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:35,913 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:35,914 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:35,914 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:35,917 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 11:36:35,922 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:35,925 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1540180324] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:35,925 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:35,927 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:35,927 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 11:36:35,927 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:35,927 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 11:36:35,933 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 11:36:35,934 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@782d80c0} 11:36:35,934 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 11:36:35,935 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:35,935 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:35,936 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 11:36:35,937 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 11:36:35,937 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 11:36:35,937 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-891978578] 11:36:35,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-891978578] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:35,937 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Shard created, persistent : true 11:36:35,937 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:35,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-2086438406 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 11:36:35,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-2086438406] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:35,938 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-default-config_testGetPrefixShardRole: Starting recovery with journal batch size 1 11:36:35,941 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 11:36:35,943 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1119729314] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:35,944 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:35,944 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:35,945 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:35,945 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 11:36:35,945 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:35,945 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 11:36:35,945 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal open: applyTo=0 11:36:35,946 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:35,946 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:35,949 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 11:36:35,950 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@6c56da6b} 11:36:35,950 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 11:36:35,950 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:35,950 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 11:36:35,950 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 11:36:35,951 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 11:36:35,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 11:36:35,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#2089793900] 11:36:35,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 11:36:35,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 11:36:35,951 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#2089793900] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:35,951 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 11:36:35,951 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:35,951 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 11:36:35,951 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Shard created, persistent : true 11:36:35,952 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#906755433 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 11:36:35,952 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#906755433] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:35,952 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-default-oper_testGetPrefixShardRole: Starting recovery with journal batch size 1 11:36:35,953 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal open: applyTo=0 11:36:35,954 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:35,954 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:35,954 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 11:36:35,954 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:35,955 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 11:36:35,955 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:35,955 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 11:36:35,955 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:35,955 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 11:36:35,955 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 11:36:35,955 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 11:36:35,955 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 11:36:35,955 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@406abbdb 11:36:35,956 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 11:36:35,956 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 11:36:35,956 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 11:36:35,956 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 11:36:35,956 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 11:36:35,956 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 11:36:35,956 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:35,956 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 11:36:35,958 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:35,959 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 11:36:35,959 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 11:36:35,959 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c7db6d9 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 11:36:35,959 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 11:36:36,1 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1540180324] 11:36:36,1 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 11:36:36,1 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-891978578] 11:36:36,2 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 11:36:36,3 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 11:36:36,3 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 11:36:36,3 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@76c4ab03 11:36:36,3 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@76c4ab03 11:36:36,3 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Stopping Shard 11:36:36,4 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal closed 11:36:36,4 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:36,4 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:36,4 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 11:36:36,5 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 11:36:36,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1540180324] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1540180324] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,5 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 11:36:36,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f0a04803690] to Actor[pekko://cluster-test/user/$a#-312197549] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-312197549] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1540180324] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1540180324] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,5 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@698e85ec 11:36:36,5 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@698e85ec 11:36:36,6 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Stopping Shard 11:36:36,6 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal closed 11:36:36,6 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:36,7 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:36,8 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:36,8 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:36,10 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:36,26 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:36,38 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-2781611402756556642] 11:36:36,38 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:36,40 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:36,40 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:36,41 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:36,41 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:36,54 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,56 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:36,56 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:36,56 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:36,57 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,60 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,66 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#81182571] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,66 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:36,66 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:36,71 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:36,71 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,72 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:36,72 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,72 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,75 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:36,75 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@52fab8c4} 11:36:36,75 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,76 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#328581077] 11:36:36,77 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,78 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,78 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,80 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,81 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,81 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,82 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:36,82 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,82 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:36,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#328581077] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,82 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-830819845 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-251019644] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,82 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-830819845] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-267894524 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,84 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,84 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,84 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,84 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,87 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-267894524] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,88 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:36,88 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,89 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,89 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:36,89 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,89 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,90 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,90 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,90 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,90 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,91 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:36,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@169b5a08} 11:36:36,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,92 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 11:36:36,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:36,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,92 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,95 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,95 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,95 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,95 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.141 ms, resetting timer for the next recovery snapshot 11:36:36,96 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2132395246] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,96 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,97 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#494567421 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,97 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#494567421] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,97 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,98 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,98 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,98 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2132395246] 11:36:36,98 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,99 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,99 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,99 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#548115792] 11:36:36,101 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,101 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.959 ms, resetting timer for the next recovery snapshot 11:36:36,101 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#548115792] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,101 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:36,101 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,101 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,101 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,102 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,102 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,102 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,102 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#518976641 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,102 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,102 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#518976641] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,103 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,103 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,104 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,104 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:36,106 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,106 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,106 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,107 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,107 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,107 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,107 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,107 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,107 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,107 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:36,107 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,108 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,108 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,108 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,110 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,110 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,110 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,111 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,111 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,111 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,113 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.863 ms, resetting timer for the next recovery snapshot 11:36:36,114 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.336 ms, resetting timer for the next recovery snapshot 11:36:36,118 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,118 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,118 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:36,118 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,118 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,118 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,119 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,120 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1469238750478456214] 11:36:36,121 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:36,121 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:36,121 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,121 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,121 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,121 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,121 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,121 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,123 AM [cluster-test-pekko.actor.default-dispatcher-7] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:36,123 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:36,123 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:36,123 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:36,139 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,141 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,144 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,155 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#630296792] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,156 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:36,156 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:36,160 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:36,160 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,161 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,163 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:36,171 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@20a4f2f8} 11:36:36,171 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 11:36:36,171 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,172 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:36,172 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,172 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,173 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,173 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,174 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,175 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,175 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,175 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,176 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,176 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:36,176 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:36,176 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1188770367] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,176 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1877419400] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,176 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-750828144] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,176 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1801929087 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,177 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-139990296] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,180 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,181 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:36,181 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,181 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,181 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-139990296 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,182 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1801929087] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,185 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,185 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,185 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,185 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,186 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:36,186 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@16fad483} 11:36:36,187 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 11:36:36,187 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:36,187 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,187 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.630 ms, resetting timer for the next recovery snapshot 11:36:36,189 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,195 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,195 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,195 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,196 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,196 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1479788877] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,196 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1502141616 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,197 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1502141616] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,197 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,197 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,198 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,198 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,198 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,199 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,199 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1324842575] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,199 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,199 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#163786757 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,199 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,199 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#163786757] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,200 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,200 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.617 ms, resetting timer for the next recovery snapshot 11:36:36,201 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,201 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,201 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,202 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,202 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:36,202 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,203 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,203 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,203 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,203 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,204 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,204 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,204 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,204 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:36:36,204 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,205 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,206 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,206 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,206 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,206 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,206 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,206 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,207 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,207 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,207 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,207 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,207 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,208 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.572 ms, resetting timer for the next recovery snapshot 11:36:36,209 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.009 ms, resetting timer for the next recovery snapshot 11:36:36,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,212 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,212 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,212 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,212 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,213 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:36,213 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,213 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:36:36,213 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,213 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,213 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,213 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,213 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,213 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,218 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:36,222 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:36,226 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:36,226 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:36,226 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:36,226 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,226 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,227 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,238 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [2896817575645857903] 11:36:36,238 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:36,240 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:36,240 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:36,245 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:36,245 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:36,251 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,253 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,254 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,264 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#99166270] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,265 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:36,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:36,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:36,268 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,268 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,274 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:36,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4ec18da2} 11:36:36,274 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 11:36:36,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:36,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,275 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,276 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:36,279 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,283 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,283 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#24600829] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,283 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-379899031] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,284 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-528410986 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,284 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,284 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-528410986] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,284 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:36,284 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:36,284 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#925455354] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,284 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,285 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1156124706 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,285 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,285 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1156124706] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,286 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,286 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,286 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,287 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,287 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,287 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,287 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,289 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,289 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,289 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,290 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:36,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,291 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,291 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,291 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,292 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:36,292 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,293 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.626 ms, resetting timer for the next recovery snapshot 11:36:36,293 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 4.080 ms, resetting timer for the next recovery snapshot 11:36:36,295 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5e6454b0} 11:36:36,296 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 11:36:36,297 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:36,297 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:36,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,300 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,300 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,301 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064438939] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,302 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,302 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,302 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:36:36,303 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,303 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1147608117] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,303 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,303 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,303 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,303 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,303 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,303 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,303 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,304 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1134638112 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,304 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1134638112] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,305 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,305 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:36,306 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 11:36:36,306 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:36,306 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#108704209 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,306 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#108704209] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:36,307 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:36,307 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,307 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,307 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,307 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:36,307 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,307 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,307 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,307 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,308 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,308 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,308 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,309 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,309 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,311 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 11:36:36,312 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 11:36:36,312 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,312 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,312 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:36,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:36,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:36,313 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:36,316 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.135 ms, resetting timer for the next recovery snapshot 11:36:36,316 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 11:36:36,316 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 11:36:36,316 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 11:36:36,319 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,319 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,319 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,320 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:36:36,320 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,320 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,320 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,320 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:36,320 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 4.259 ms, resetting timer for the next recovery snapshot 11:36:36,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:36,324 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:36,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:36,324 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 11:36:36,324 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 11:36:36,324 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,324 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:36,324 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 11:36:37,54 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:37,54 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:37,55 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,55 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,56 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,63 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,64 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,98 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#81182571] 11:36:37,98 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:37,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,98 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,100 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#81182571] 11:36:37,100 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:37,100 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,100 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,100 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 11:36:37,100 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,100 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,100 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:36:37,101 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:36:37,101 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:36:37,101 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:36:37,101 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:37,101 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:36:37,101 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:37,102 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:36:37,103 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,103 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,103 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,104 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,109 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 11:36:37,109 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 11:36:37,110 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:37,110 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:37,110 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:37,110 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:37,110 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:37,110 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:37,110 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 11:36:37,110 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 11:36:37,110 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 11:36:37,110 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 11:36:37,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 11:36:37,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 11:36:37,110 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:37,110 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:37,110 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:37,110 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:36:37,110 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:37,110 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:37,110 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 11:36:37,111 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 11:36:37,111 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 11:36:37,111 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 11:36:37,111 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,111 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,112 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,112 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,112 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,112 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,112 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,112 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:37,119 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:37,119 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:36:37,119 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,119 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 11:36:37,119 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@cc2552d 11:36:37,120 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,120 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 11:36:37,120 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:36:37,120 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 11:36:37,120 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 11:36:37,121 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 11:36:37,121 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@216b021 11:36:37,121 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:37,121 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,121 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,122 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:36:37,122 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,122 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 11:36:37,122 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,122 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,122 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@28461dfd 11:36:37,122 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38e75590 11:36:37,123 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:37,123 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,123 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:36:37,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 11:36:37,123 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,123 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:36:37,124 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 11:36:37,124 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 11:36:37,124 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@746547bc 11:36:37,124 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3e956d3a 11:36:37,124 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 11:36:37,124 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@b3ef958 11:36:37,125 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:37,125 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 11:36:37,125 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 11:36:37,126 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 11:36:37,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 11:36:37,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:36:37,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 11:36:37,126 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,126 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@30529498 11:36:37,127 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,127 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,127 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,127 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,127 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,127 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,127 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69622911 11:36:37,127 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 11:36:37,127 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#81182571] 11:36:37,128 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:37,128 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@308469e9 11:36:37,128 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,128 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,129 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@e26e628 11:36:37,129 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,129 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,129 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,130 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:37,131 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 11:36:37,131 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 11:36:37,131 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d21c8ac 11:36:37,131 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,131 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:36:37,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,131 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 11:36:37,131 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:37,131 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 11:36:37,132 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:36:37,132 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,132 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 11:36:37,132 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 11:36:37,132 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#81182571] 11:36:37,133 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:36:37,133 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#328581077] 11:36:37,133 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#328581077] 11:36:37,134 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-251019644] 11:36:37,134 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:37,134 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2132395246] 11:36:37,134 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:37,134 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:36:37,134 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,134 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2132395246] 11:36:37,134 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 11:36:37,135 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 11:36:37,135 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,135 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-251019644] 11:36:37,135 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:36:37,135 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#548115792] 11:36:37,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#548115792] 11:36:37,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#630296792] 11:36:37,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:37,138 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1188770367] 11:36:37,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1188770367] 11:36:37,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#630296792] 11:36:37,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:36:37,138 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1877419400] 11:36:37,138 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1877419400] 11:36:37,138 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:37,138 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 11:36:37,138 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:36:37,139 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 11:36:37,139 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 11:36:37,139 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,141 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-750828144] 11:36:37,141 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:37,141 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1479788877] 11:36:37,141 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1479788877] 11:36:37,142 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-750828144] 11:36:37,142 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 11:36:37,142 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1324842575] 11:36:37,142 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1324842575] 11:36:37,143 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#99166270] 11:36:37,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:37,143 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#24600829] 11:36:37,146 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#24600829] 11:36:37,146 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#99166270] 11:36:37,146 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:36:37,146 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#925455354] 11:36:37,146 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#925455354] 11:36:37,146 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-379899031] 11:36:37,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:37,147 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064438939] 11:36:37,147 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064438939] 11:36:37,147 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-379899031] 11:36:37,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:36:37,147 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1147608117] 11:36:37,147 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1147608117] 11:36:37,148 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#81182571] 11:36:37,148 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:37,148 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,148 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-611688216] 11:36:37,149 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-251019644] 11:36:37,149 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:37,149 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2132395246] 11:36:37,149 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2132395246] 11:36:37,150 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 11:36:37,150 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,150 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,150 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,150 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,151 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,151 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,151 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,151 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,151 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 11:36:37,151 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:37,152 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:37,152 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:37,152 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,152 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,152 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:36:37,153 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,153 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,153 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,153 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,153 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:37,153 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:37,154 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:37,154 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:37,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:37,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:37,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:37,159 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:37,161 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:37,176 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:37,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,177 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:36:37,177 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,177 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,179 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,179 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,179 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:37,179 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:37,179 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:37,180 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,180 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:37,180 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:37,180 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,180 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:37,181 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,181 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:37,181 AM [cluster-test-pekko.actor.default-dispatcher-7] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:37,181 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:37,182 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:37,182 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:38,74 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:38,173 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [2896817575645857903] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:38,194 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1469238750478456214] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:38,198 AM [cluster-test-pekko.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:38,199 AM [cluster-test-pekko.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:38,200 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:38.202] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:38,206 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:38,207 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:38,207 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 11:36:38,207 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:36:38,207 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@67fc7392 11:36:38,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@67fc7392 11:36:38,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 11:36:38,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35efa8e 11:36:38,208 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@52ee906 11:36:38,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35efa8e 11:36:38,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 11:36:38,208 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f84f514 11:36:38,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 11:36:38,724 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 516.5 ms 11:36:38,724 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 11:36:38,725 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:38,725 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:38,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 11:36:38,734 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 526.5 ms 11:36:38,734 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 11:36:38,735 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:38,735 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:38,736 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:38,736 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:38,736 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 11:36:38,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:38,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 11:36:38,737 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:38,737 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c971a5f 11:36:38,737 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2e482e6c 11:36:38,737 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d514bd 11:36:38,737 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@626253dd 11:36:38,737 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2e482e6c 11:36:38,737 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d514bd 11:36:38,737 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 11:36:38,737 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 11:36:39,43 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 11:36:39,44 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 306.6 ms 11:36:39,44 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 11:36:39,45 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:39,45 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:39,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 11:36:39,54 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 316.5 ms 11:36:39,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 11:36:39,54 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 11:36:39,55 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 11:36:39,55 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:39,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:39,56 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:39,56 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:39,63 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:39.064] [cluster-test-pekko.actor.internal-dispatcher-8] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:39,91 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:39,103 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1765925482822173207] 11:36:39,104 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:39,107 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:39,107 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:39,108 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:39,108 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:39,119 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,120 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,122 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:39,122 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:39,122 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:39,124 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,127 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#744440399] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,129 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:39,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:39,133 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:39,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,134 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:39,134 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,134 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,136 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@585e514e} 11:36:39,137 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 11:36:39,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:39,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,138 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1795225158] 11:36:39,138 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,139 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,139 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,140 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,140 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#397669705 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,140 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#397669705] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,140 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,141 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,141 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#801236708 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,141 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1795225158] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,141 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#801236708] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,141 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,142 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,144 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,144 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,145 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#640484071] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,145 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:39,146 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:39,147 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:39,148 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,148 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,148 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:39,148 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,148 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,150 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,150 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,150 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,150 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,151 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,151 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,151 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,151 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,151 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@1a2710d7} 11:36:39,152 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 11:36:39,152 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:39,152 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#273599321] 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,153 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.997 ms, resetting timer for the next recovery snapshot 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1629809009] 11:36:39,153 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,153 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#273599321] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,154 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-740496939 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,154 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,154 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-740496939] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,156 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,156 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1629809009] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,156 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,157 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1935400240 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,157 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1935400240] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,157 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,157 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:39,157 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,158 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,158 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,158 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,158 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,159 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,159 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,159 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,159 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,160 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,160 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,160 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,162 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,162 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,163 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,163 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:39,163 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,163 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,163 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,166 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,166 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,166 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,166 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.340 ms, resetting timer for the next recovery snapshot 11:36:39,166 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 7.741 ms, resetting timer for the next recovery snapshot 11:36:39,171 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,171 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,171 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,171 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:39,171 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,171 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,171 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,172 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-3490143982058026] 11:36:39,172 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:39,173 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,173 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,173 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,173 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:39,173 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,173 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,173 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 7.587 ms, resetting timer for the next recovery snapshot 11:36:39,174 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,177 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:39,177 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:39,182 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:39,182 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:39,177 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,177 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:39,186 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,186 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,186 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,187 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,187 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,192 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,196 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,198 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,203 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1895358678] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,206 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:39,206 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:39,207 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:39,207 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,208 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,215 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,216 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@1e6b6717} 11:36:39,216 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 11:36:39,217 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,218 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,219 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:39,219 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,220 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,220 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,220 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,221 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#726683528] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,221 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-611711336] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,221 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2131440036 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,221 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1755949475] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,221 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2131440036] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,221 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,222 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,223 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,224 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1080831286] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,224 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,224 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,224 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,224 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,227 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1755949475 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,227 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,228 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,228 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,228 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,228 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,229 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,229 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,230 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,230 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 11:36:39,232 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 4.430 ms, resetting timer for the next recovery snapshot 11:36:39,234 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:39,235 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:39,235 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,235 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,235 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,236 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:39,236 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,236 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@3acbf5f6} 11:36:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,238 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:39,238 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,239 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,239 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-337657178] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,239 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 4.081 ms, resetting timer for the next recovery snapshot 11:36:39,240 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,240 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-22571524 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,240 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-22571524] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,241 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,242 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1821607815] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,242 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,242 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-424859762 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,242 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-424859762] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,243 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:36:39,243 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,243 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,243 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,243 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,243 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,243 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,243 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,243 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,243 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,243 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,244 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,244 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,244 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,244 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,244 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,242 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,247 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,247 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,247 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,249 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,250 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.593 ms, resetting timer for the next recovery snapshot 11:36:39,254 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,254 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,254 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,254 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:39,254 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,254 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,254 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,255 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,255 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,255 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,255 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,258 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,258 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,258 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,261 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.118 ms, resetting timer for the next recovery snapshot 11:36:39,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:39,266 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,266 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,266 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,266 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:36:39,266 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,266 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,266 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,267 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,268 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,269 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:39,278 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-6502577415054238917] 11:36:39,278 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:39,279 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:39,279 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:39,281 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:39,281 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:39,288 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,289 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,291 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,300 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#858430020] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,300 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:39,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:39,303 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:39,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,304 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5668dd83} 11:36:39,306 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 11:36:39,306 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,308 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:39,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:39,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,311 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,311 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1708924818] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,313 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,314 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,315 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2119204173 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,315 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2119204173] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,316 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2138403375] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,316 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,316 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,317 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#7556743 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,317 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#7556743] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,317 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,318 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:39,320 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,324 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:39,324 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,324 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,324 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,324 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,324 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:39,328 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-955362696] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,329 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,329 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,329 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,329 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,329 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,329 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,329 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,329 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,330 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:39,330 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:39,331 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:39,331 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,331 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,331 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:39,332 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,332 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,334 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,335 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 11:36:39,335 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@ccb24fa} 11:36:39,335 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:39,335 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:39,335 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,335 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,335 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,336 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,336 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,336 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,336 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1407191157] 11:36:39,337 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1407191157] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,337 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,337 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-304018072 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,337 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-304018072] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,337 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,338 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,338 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,339 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,339 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,339 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-733040809] 11:36:39,339 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,340 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-733040809] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,340 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,340 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,340 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,341 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 5.193 ms, resetting timer for the next recovery snapshot 11:36:39,341 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 11:36:39,341 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1417642450 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,342 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1417642450] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:39,342 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,342 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,342 AM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,343 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 11:36:39,345 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 11:36:39,345 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,345 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,345 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 11:36:39,346 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:36:39,346 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,346 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,346 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 11:36:39,346 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,346 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:39,346 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 11:36:39,346 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,346 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,346 AM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 4.054 ms, resetting timer for the next recovery snapshot 11:36:39,350 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 11:36:39,350 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 11:36:39,350 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,350 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 11:36:39,350 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,350 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,350 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,350 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:39,350 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,350 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,350 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,350 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 5.119 ms, resetting timer for the next recovery snapshot 11:36:39,354 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,354 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,354 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 4.175 ms, resetting timer for the next recovery snapshot 11:36:39,354 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,354 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,354 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:39,354 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,354 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,355 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,358 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:36:39,358 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 11:36:39,358 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,358 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,358 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 11:36:39,358 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,358 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:39,358 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 11:36:39,533 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 11:36:39,533 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:39,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,533 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 11:36:39,533 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 11:36:39,561 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 11:36:39,561 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:39,561 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,561 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,561 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 11:36:39,562 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 11:36:39,571 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 11:36:39,572 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:39,572 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,572 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,572 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 11:36:39,572 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 11:36:39,572 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 11:36:39,573 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:36:39,573 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 11:36:39,573 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:39,573 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 11:36:39,573 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 11:36:39,913 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 11:36:39,922 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 11:36:39,932 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 11:36:39,992 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 11:36:40,120 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:40,120 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:40,120 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:40,121 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,121 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,128 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,129 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,205 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,293 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:36:40,303 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 11:36:40,304 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,304 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,313 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:40,313 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 11:36:40,313 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,313 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 11:36:40,313 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 11:36:40,313 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18e7853e 11:36:40,313 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 11:36:40,314 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@670d7f5b 11:36:40,314 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,314 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,314 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18660a2a 11:36:40,315 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,315 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,319 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 11:36:40,319 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,319 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 11:36:40,319 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 11:36:40,321 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 11:36:40,322 AM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,322 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,327 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:40,327 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 11:36:40,327 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,327 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 11:36:40,327 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 11:36:40,327 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40ffb391 11:36:40,327 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 11:36:40,328 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,328 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,328 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a16a9ab 11:36:40,329 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,329 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6040b5df 11:36:40,331 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 11:36:40,331 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,331 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 11:36:40,331 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 11:36:40,331 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 11:36:40,331 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,333 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,333 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,339 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:40,339 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 11:36:40,339 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,339 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 11:36:40,339 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 11:36:40,340 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b466366 11:36:40,340 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 11:36:40,340 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,340 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,340 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@390144ef 11:36:40,340 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 11:36:40,341 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,341 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,341 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7787f788 11:36:40,341 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 11:36:40,343 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 11:36:40,343 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,343 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 11:36:40,343 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 11:36:40,343 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 11:36:40,403 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 11:36:40,405 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,405 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 11:36:40,411 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:36:40,412 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 11:36:40,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 11:36:40,412 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 11:36:40,412 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60cb6a97 11:36:40,412 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 11:36:40,413 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,413 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f146b34 11:36:40,413 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 11:36:40,413 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,414 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b12441b 11:36:40,414 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 11:36:40,414 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 11:36:40,414 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 11:36:40,417 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 11:36:40,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 11:36:40,418 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 11:36:40,418 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 11:36:40,418 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 11:36:40,418 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 11:36:40,418 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:40,418 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 11:36:40,418 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#744440399] 11:36:40,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:40,418 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:40,418 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:40,419 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 11:36:40,419 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,419 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,419 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:36:40,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 11:36:40,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:36:40,420 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:40,420 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:36:40,420 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 11:36:40,420 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:40,420 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:36:40,421 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,421 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,421 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 11:36:40,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 11:36:40,422 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 11:36:40,424 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,426 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 11:36:40,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 11:36:40,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 11:36:40,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#744440399] 11:36:40,427 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:40,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:40,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:40,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#744440399] 11:36:40,427 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 11:36:40,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1795225158] 11:36:40,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1795225158] 11:36:40,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#640484071] 11:36:40,428 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 11:36:40,428 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#273599321] 11:36:40,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#273599321] 11:36:40,429 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#640484071] 11:36:40,429 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 11:36:40,429 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1629809009] 11:36:40,429 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1629809009] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1895358678] 11:36:40,430 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:40,430 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#726683528] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#726683528] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1895358678] 11:36:40,430 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 11:36:40,430 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-611711336] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-611711336] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1080831286] 11:36:40,430 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 11:36:40,430 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-337657178] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-337657178] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1080831286] 11:36:40,430 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 11:36:40,430 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1821607815] 11:36:40,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1821607815] 11:36:40,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#858430020] 11:36:40,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:40,431 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1708924818] 11:36:40,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1708924818] 11:36:40,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#858430020] 11:36:40,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 11:36:40,431 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2138403375] 11:36:40,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2138403375] 11:36:40,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-955362696] 11:36:40,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 11:36:40,431 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1407191157] 11:36:40,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1407191157] 11:36:40,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-955362696] 11:36:40,432 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 11:36:40,432 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-733040809] 11:36:40,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-733040809] 11:36:40,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#744440399] 11:36:40,432 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:40,432 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:40,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#545723672] 11:36:40,433 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:40,433 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,433 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,433 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 11:36:40,433 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,433 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,434 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,434 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,434 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:40,434 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:40,434 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:40,434 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,434 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,434 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:36:40,435 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,435 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,435 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,435 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,435 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:40,435 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:40,436 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:40,436 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:40,439 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:40,439 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:40,440 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:40,440 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:40,441 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:40,454 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:40,454 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,454 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,455 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:36:40,455 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,455 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,456 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,456 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,457 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:40,457 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:40,457 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:40,457 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,457 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:40,457 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:40,458 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,458 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:40,459 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,459 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:40,459 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:40,460 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:40,461 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:40,461 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:41,38 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 11:36:41,38 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 11:36:41,39 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 11:36:41,39 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 11:36:41,139 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:41,449 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-6502577415054238917] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:41,479 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-3490143982058026] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:41,480 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:41,480 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:41,481 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:41.484] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:41,497 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:41,498 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:41,498 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 11:36:41,498 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:36:41,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73932959 11:36:41,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73932959 11:36:41,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 11:36:41,498 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c41dee8 11:36:41,498 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c41dee8 11:36:41,498 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 11:36:41,498 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2395db81 11:36:41,498 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@58f326f1 11:36:41,928 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 11:36:41,929 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 430.7 ms 11:36:41,929 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 11:36:41,929 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:41,930 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:42,9 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 11:36:42,9 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 510.8 ms 11:36:42,9 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 11:36:42,10 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:42,10 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:42,11 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:42,11 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:42,11 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 11:36:42,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:42,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 11:36:42,12 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:42,12 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@73bc7f06 11:36:42,12 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f6994f 11:36:42,12 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c6c4755 11:36:42,12 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c6c4755 11:36:42,12 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 11:36:42,13 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@115d5417 11:36:42,13 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@115d5417 11:36:42,13 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 11:36:42,559 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 11:36:42,559 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 546.5 ms 11:36:42,559 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 11:36:42,559 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:42,560 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:42,579 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 11:36:42,579 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 566.7 ms 11:36:42,579 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 11:36:42,580 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 11:36:42,581 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 11:36:42,581 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:42,581 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:42,583 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:42,583 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:42,586 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:42.588] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:42,616 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:42,630 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [6779848772413911447] 11:36:42,630 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:42,630 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:42,630 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:42,631 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:42,631 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:42,641 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,646 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,651 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 11:36:42,659 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:42,659 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:42,659 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:42,665 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1426176352] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,666 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:42,667 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:42,668 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:42,668 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,669 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,669 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:42,670 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,670 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,671 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:42,672 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@5c97e83b} 11:36:42,672 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 11:36:42,673 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,675 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,678 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,680 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:42,680 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,681 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#1859461364] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,682 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:42,682 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:42,683 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,684 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-99700907 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,684 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 11:36:42,684 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,684 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,684 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833] 11:36:42,684 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,684 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,685 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,684 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,685 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,685 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-1215201564] 11:36:42,685 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:42,685 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,685 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#719349161 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,685 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,685 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,686 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:42,686 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,686 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,687 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,688 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-99700907] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,688 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-1215201564] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,688 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#719349161] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,691 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 11:36:42,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@2424dc7d} 11:36:42,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:42,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,695 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#1561586450] 11:36:42,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,695 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,695 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#1561586450] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,695 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,696 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,696 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,696 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-446282168 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,696 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-446282168] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,697 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,697 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,698 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,698 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,698 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-667669277 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,698 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,699 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,699 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,699 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,701 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,701 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,701 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:42,701 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 11:36:42,701 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,701 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,701 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,707 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1696443428] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,707 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-667669277] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,707 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,708 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,708 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,708 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,709 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,709 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,709 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 11:36:42,709 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 11:36:42,710 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,711 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,711 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,711 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,738 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:42,738 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 11:36:42,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,736 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1696443428] 11:36:42,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:42,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 11:36:42,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:42,739 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 11:36:42,739 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,740 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,740 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,740 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,740 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 11:36:42,740 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 11:36:42,740 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 11:36:42,740 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 11:36:42,743 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:42,760 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1071050117282116672] 11:36:42,760 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:42,764 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:42,765 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:42,767 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:42,767 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:42,780 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,782 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,786 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 11:36:42,795 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-498004092] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,795 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:42,795 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:42,801 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:42,802 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,802 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:42,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:42,805 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@3e274e21} 11:36:42,807 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 11:36:42,807 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:42,807 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,807 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,808 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:42,808 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,809 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,811 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1578656673 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,811 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,811 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1578656673] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,812 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,813 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#1952373538] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,813 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,813 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1363282822] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,814 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-289300720 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:42,814 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-289300720] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,814 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,814 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:42,814 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:42,815 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,819 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:42,819 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,819 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,819 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,819 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,822 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 11:36:42,822 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,822 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@16da92d2} 11:36:42,823 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 11:36:42,823 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,823 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:42,823 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,824 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,824 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,824 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#1974100755] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,825 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#547122785 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,825 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#547122785] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,825 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,826 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#1825552914] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,826 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,826 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1832671262 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,826 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1832671262] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,826 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,827 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,827 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,827 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,828 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,828 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,828 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:36:42,828 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,828 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 11:36:42,828 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,828 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 11:36:42,828 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,828 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 11:36:42,830 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,830 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,830 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,830 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,831 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,831 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,831 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:42,831 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 11:36:42,831 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,831 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 11:36:42,834 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:42,837 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 11:36:42,837 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,837 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,837 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,838 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,839 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,839 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,840 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,840 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,846 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 11:36:42,846 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:36:42,846 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 11:36:42,846 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,846 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 11:36:42,846 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 11:36:42,847 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:42,847 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,847 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:42,847 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,847 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,847 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:36:42,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-8095986206250214912] 11:36:42,859 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:42,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:42,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:42,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:42,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:42,871 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,872 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,875 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 11:36:42,879 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:42,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:42,881 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1175123807] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,894 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:42,896 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:42,903 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:42,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:42,904 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:42,904 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,904 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,906 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:42,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@577eee1a} 11:36:42,907 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 11:36:42,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:42,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,907 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-3-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#722396026] 11:36:42,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:42,908 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:42,910 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,910 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#722396026] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,911 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#1590160974 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,911 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,911 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#1590160974] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,911 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,912 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,912 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:42,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-3-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1928128607] 11:36:42,912 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,912 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1928128607] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,913 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#1413349291] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,913 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,913 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:42,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:42,914 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,914 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#580636210 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:42,915 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#580636210] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,915 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,916 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,917 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 11:36:42,917 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 11:36:42,917 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 11:36:42,921 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,921 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:42,921 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,921 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,922 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 11:36:42,922 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 11:36:42,923 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:42,922 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,923 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:42,923 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,923 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@a0a12b7} 11:36:42,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:42,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,923 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-3-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#2111453979] 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,924 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:36:42,924 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,924 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-3-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#2075451811] 11:36:42,924 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,924 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,924 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,924 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:42,924 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 11:36:42,925 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,925 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#2075451811] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,926 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#2111453979] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,925 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 11:36:42,927 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#-319569697 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,927 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#-319569697] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,927 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,928 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,928 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 11:36:42,928 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 11:36:42,928 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,928 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1985362797] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:42,929 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1985362797 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,929 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,930 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 11:36:42,931 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:42,931 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:42,931 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,931 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:36:42,931 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 11:36:42,931 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,931 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,931 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,931 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:42,931 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,932 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 11:36:42,936 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 11:36:42,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:42,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 11:36:42,937 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:42,937 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:42,937 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:42,937 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 11:36:42,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 11:36:42,937 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 11:36:43,36 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 11:36:43,37 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:43,37 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:43,37 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:43,37 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 11:36:43,37 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 11:36:43,37 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 11:36:43,37 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:43,37 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:43,37 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:43,37 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 11:36:43,38 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 11:36:43,56 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 11:36:43,56 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:43,56 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:43,56 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:43,56 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 11:36:43,56 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 11:36:43,115 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 11:36:43,116 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:43,116 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:43,116 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:43,116 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 11:36:43,116 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 11:36:43,426 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 11:36:43,456 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 11:36:43,465 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 11:36:43,527 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 11:36:43,645 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:43,645 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:43,645 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,645 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,646 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:43,646 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:43,652 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,652 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,767 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 11:36:43,769 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,769 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,779 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:43,779 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:43,779 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 11:36:43,780 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 11:36:43,780 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7855b506 11:36:43,780 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 11:36:43,781 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 11:36:43,781 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f0212f7 11:36:43,785 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 11:36:43,786 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,786 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d2452a7 11:36:43,787 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:43,791 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:43,792 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:43,792 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:43,792 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 11:36:43,792 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 11:36:43,792 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75334013 11:36:43,792 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c896d27 11:36:43,792 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 11:36:43,792 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 11:36:43,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 11:36:43,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d47539c 11:36:43,815 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 11:36:43,817 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,818 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,824 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:43,824 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:43,824 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 11:36:43,825 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 11:36:43,825 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7692547c 11:36:43,825 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 11:36:43,825 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 11:36:43,825 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b720b15 11:36:43,825 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 11:36:43,825 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 11:36:43,826 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 11:36:43,826 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@a009573 11:36:43,827 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 11:36:43,877 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:36:43,915 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 11:36:43,917 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,917 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:43,924 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:43,924 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:43,924 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 11:36:43,924 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 11:36:43,924 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f60cd7c 11:36:43,924 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 11:36:43,924 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:43,924 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 11:36:43,924 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 11:36:43,924 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1426176352] 11:36:43,925 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6845c850 11:36:43,925 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:43,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:43,925 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833] 11:36:43,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833] 11:36:43,925 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 11:36:43,925 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@22d57fc7 11:36:43,925 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:43,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-498004092] 11:36:43,926 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:43,926 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1175123807] 11:36:43,926 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:43,926 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#722396026] 11:36:43,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#722396026] 11:36:43,927 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-1}, datastoreType Config 11:36:43,927 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 11:36:43,927 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:36:43,928 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@291812aa] 11:36:43,928 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,928 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 11:36:43,928 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 11:36:43,929 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 11:36:43,929 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-498004092] 11:36:43,929 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:43,929 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,929 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,934 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:43,934 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833] with 10000 ms timeout 11:36:43,934 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 11:36:43,934 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 11:36:43,935 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72adde76 11:36:43,935 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72adde76 11:36:43,935 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 11:36:43,936 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 11:36:43,937 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 11:36:43,938 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f2762dd 11:36:43,939 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1c20e3f5 11:36:43,939 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 11:36:43,941 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3477f0fb 11:36:43,941 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,943 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 11:36:43,943 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:36:43,943 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:43,943 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:43,943 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 11:36:43,943 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c2c5e8d 11:36:43,943 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 11:36:43,944 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote{term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than follower's term 3 - updating term 11:36:43,950 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:36:43,950 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:43,950 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 11:36:43,950 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 11:36:43,950 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@717cbadd 11:36:43,950 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 11:36:43,950 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:43,951 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 11:36:43,951 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@210fef6b 11:36:43,951 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:43,952 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 11:36:43,983 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,983 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-498004092] 11:36:43,984 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:43,984 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-140587115] 11:36:43,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1175123807] 11:36:43,984 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:43,984 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#722396026] 11:36:43,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#722396026] 11:36:43,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1426176352] 11:36:43,985 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:43,985 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 11:36:43,985 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:43,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 11:36:43,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 11:36:43,986 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 11:36:43,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f1a6f82 11:36:43,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f1a6f82 11:36:43,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 11:36:43,987 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@769b0180 11:36:43,987 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 11:36:43,987 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal closed 11:36:43,989 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@409f1081 11:36:43,990 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 11:36:43,994 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 5 11:36:43,994 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:36:43,994 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:43,994 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:43,995 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 11:36:43,995 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55af4cb1 11:36:43,995 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 11:36:43,995 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "RequestVote{term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 11:36:44,1 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:36:44,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:44,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:44,1 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 11:36:44,1 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3fcc15fd 11:36:44,1 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 11:36:44,2 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:36:44,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:44,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 11:36:44,2 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 11:36:44,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c244bd4 11:36:44,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 11:36:44,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:44,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader changed to member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:44,2 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to member-2-shard-cars-config_testRemoveShardLeaderReplica in 16.56 ms 11:36:44,3 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@9a63c61 11:36:44,3 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 11:36:44,3 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 11:36:44,3 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 11:36:44,3 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal closed 11:36:44,4 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:44,4 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:44,4 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 11:36:44,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 11:36:44,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 11:36:44,4 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:36:44,4 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 11:36:44,4 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 11:36:44,4 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 11:36:44,5 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal closed 11:36:44,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:44,5 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:44,6 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:44,6 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:44,11 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:44,11 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:44,11 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:44,11 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:44,13 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:44,28 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:44,28 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 11:36:44,28 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 11:36:44,28 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:36:44,29 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3b40f8e6 11:36:44,29 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3b40f8e6 11:36:44,29 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 11:36:44,29 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@162815e3 11:36:44,29 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 11:36:44,29 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal closed 11:36:44,95 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 11:36:44,96 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 11:36:44,145 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 11:36:44,146 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:44,243 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 5 11:36:44,244 AM [cluster-test-shard-dispatcher-16] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "AppendEntriesReply{term=5, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 11:36:44,248 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:36:44,248 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 11:36:44,248 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:44,248 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 11:36:44,248 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70486951 11:36:44,248 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 11:36:44,248 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 11:36:44,415 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:44,465 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:44,663 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:44,735 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:44,784 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:45,22 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-8095986206250214912] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:45,25 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-8095986206250214912] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:45,55 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:45,105 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:45,376 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:45,424 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:45,698 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 11:36:45,744 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:45,953 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 11:36:45,953 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.018 s 11:36:45,953 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 11:36:45,954 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal closed 11:36:45,954 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#149389833] 11:36:46,15 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:36:46,66 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:46,335 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:46,386 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:46,655 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:46,705 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:46,975 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:47,25 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:47,295 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:47,344 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:47,616 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:47,665 AM [cluster-test-shard-dispatcher-23] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:47,935 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:47,988 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 11:36:48,255 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:48,306 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 11:36:48,575 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:48,625 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:48,893 AM [cluster-test-shard-dispatcher-16] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:48,893 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:48,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:48,893 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 11:36:48,893 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:48,896 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:48,896 AM [cluster-test-shard-dispatcher-16] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 11:36:48,897 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 11:36:48,897 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:48,897 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 11:36:48,897 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 11:36:48,897 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 11:36:48,903 AM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:48,903 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:48,903 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:48,903 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 11:36:48,903 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:48,943 AM [cluster-test-shard-dispatcher-20] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:48,944 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:48,944 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:48,944 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 11:36:48,944 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:48,947 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 11:36:48,947 AM [cluster-test-shard-dispatcher-20] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 11:36:48,947 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 11:36:48,947 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:48,947 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 11:36:48,947 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 11:36:48,947 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 11:36:49,215 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 11:36:49,265 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 11:36:49,536 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:49,585 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:49,764 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:49,764 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:49,764 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 11:36:49,764 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:49,855 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:49,896 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 11:36:49,896 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:49,896 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:49,905 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:50,182 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 327, lastApplied : -1, commitIndex : -1 11:36:50,225 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:50,495 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 11:36:50,545 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:50,815 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:50,866 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:51,135 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:51,185 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:51,455 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:51,505 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:51,509 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:36:51,680 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:36:51,775 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:51,826 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:52,95 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:52,144 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 11:36:52,415 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:52,464 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:52,735 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:52,784 AM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:53,55 AM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 11:36:53,105 AM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 11:36:53,275 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 11:36:53,275 AM [cluster-test-shard-dispatcher-27] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.247 s 11:36:53,275 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 11:36:53,276 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 11:36:53,276 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal closed 11:36:53,277 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:53,277 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:53,277 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 11:36:53,278 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 11:36:53,278 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 11:36:53,278 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:53,278 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 11:36:53,278 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 11:36:53,279 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 11:36:53,279 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal closed 11:36:53,279 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:53,279 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:53,280 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:53,280 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:54,12 AM [cluster-test-shard-dispatcher-18] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:54,13 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:54,13 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:54,13 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 11:36:54,13 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:54,63 AM [cluster-test-shard-dispatcher-18] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:54,63 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 11:36:54,63 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 11:36:54,63 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 11:36:54,63 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 11:36:54,293 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1071050117282116672] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:54,299 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:54,299 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:54,301 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:54.303] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:54,306 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 11:36:54,307 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 11:36:54,307 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 11:36:54,308 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 11:36:54,308 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal closed 11:36:54,308 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:54,309 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:54,309 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 11:36:54,309 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 11:36:54,309 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 11:36:54,309 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:54,309 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 11:36:54,309 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 11:36:54,309 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal closed 11:36:54,310 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 11:36:54,310 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:54,310 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:54,311 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:54,311 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:54,313 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:54.315] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:54,339 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:54,347 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1636358201620681918] 11:36:54,347 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:54,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:54,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:54,357 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:54,357 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:54,369 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,370 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:54,371 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:54,372 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:54,374 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,378 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,382 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:54,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:54,383 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-2965194] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,384 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:54,384 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,384 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,385 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,385 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,385 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,389 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 11:36:54,389 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@51f5330e} 11:36:54,390 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 11:36:54,390 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,390 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,390 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,392 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#897913779] 11:36:54,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,392 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,392 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 11:36:54,393 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 11:36:54,394 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,394 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 11:36:54,395 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 11:36:54,395 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 11:36:54,396 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 11:36:54,397 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:54,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:54,402 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,402 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,402 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#1525758045] 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-432146619 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-975962179 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#897913779] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#1525758045] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#1231740444] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-432146619] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,403 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-975962179] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,404 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,404 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:54,404 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,404 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,405 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,405 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,405 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,405 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,405 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,410 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,411 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:54,411 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 11:36:54,411 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,411 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,411 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 11:36:54,411 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 11:36:54,411 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,412 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:54,412 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 11:36:54,412 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 11:36:54,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@264aa953} 11:36:54,412 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 11:36:54,412 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 11:36:54,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,414 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 11:36:54,414 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 11:36:54,415 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-734692976 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,415 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,415 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,415 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#1594546707] 11:36:54,415 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,415 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 11:36:54,415 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 11:36:54,415 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@667e319a 11:36:54,415 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,416 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 11:36:54,416 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 11:36:54,416 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 11:36:54,416 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e0fcd7b 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 11:36:54,416 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 11:36:54,417 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 11:36:54,417 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 11:36:54,418 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 11:36:54,418 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-9226197 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,418 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-1=false} 11:36:54,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 11:36:54,419 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,419 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,419 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1602851521] 11:36:54,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:54,419 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,420 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,420 AM [cluster-test-pekko.actor.default-dispatcher-6] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard ShardName{value=cars} java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$10.onComplete(ShardManager.java:1568) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:310) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:307) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:234) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:231) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 11:36:54,420 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-2965194] 11:36:54,420 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:54,420 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#897913779] 11:36:54,420 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#897913779] 11:36:54,421 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 11:36:54,421 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 11:36:54,421 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,421 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,421 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@669af0f7 11:36:54,421 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@669af0f7 11:36:54,421 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,421 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 11:36:54,422 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 11:36:54,422 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,422 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#1594546707] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,422 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-734692976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,422 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,422 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1602851521] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,423 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-9226197] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,423 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:36:54,423 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2893798e 11:36:54,423 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2893798e 11:36:54,423 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 11:36:54,423 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 11:36:54,424 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:54,424 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:54,424 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 11:36:54,424 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 11:36:54,425 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:54,436 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,437 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,437 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,437 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,438 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 11:36:54,438 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 11:36:54,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:54,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 11:36:54,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:54,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 11:36:54,440 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 11:36:54,440 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,440 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,440 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,441 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 11:36:54,441 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:54,441 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:54,441 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 11:36:54,441 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 11:36:54,441 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 11:36:54,441 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 11:36:54,441 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,441 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,441 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,442 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 11:36:54,442 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:36:54,442 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:54,442 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 11:36:54,442 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 11:36:54,442 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 11:36:54,443 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 11:36:54,443 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:54,443 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:54,446 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:54,446 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:54,447 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:54,467 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:36:54,482 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-5437810311091834868] 11:36:54,482 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 11:36:54,484 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 11:36:54,484 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 11:36:54,485 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:54,485 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:54,499 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,500 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:36:54,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:36:54,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 11:36:54,503 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,505 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-237656367] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,506 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 11:36:54,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 11:36:54,510 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,510 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 11:36:54,510 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,510 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,511 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,511 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,511 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@83ab204} 11:36:54,511 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 11:36:54,511 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,512 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,514 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,515 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 11:36:54,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 11:36:54,516 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,516 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,518 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,518 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-554263048 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,518 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1701901045] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,518 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#22778456] 11:36:54,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,519 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,519 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,519 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#161973109] 11:36:54,519 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,519 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1204116226 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,520 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 11:36:54,520 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,520 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,520 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,521 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,521 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,521 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,521 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,521 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,522 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#22778456] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-554263048] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#161973109] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,522 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1204116226] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,522 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,522 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,523 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@2571485b} 11:36:54,523 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 11:36:54,525 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,525 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,526 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,527 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,527 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,527 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,527 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,527 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,527 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1067319002] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,528 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1067319002] 11:36:54,528 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,528 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 11:36:54,528 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 11:36:54,528 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,528 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,528 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-212994777] 11:36:54,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,528 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,529 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,529 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-212994777] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,530 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,531 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#846965255 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,531 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,531 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,531 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#846965255] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,531 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,531 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1439884772 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,532 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,532 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1439884772] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,532 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,532 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,532 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,532 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,533 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,534 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:36:54,535 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,536 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,536 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,536 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,536 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 11:36:54,536 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 11:36:54,536 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,537 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,539 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 11:36:54,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 11:36:54,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,539 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,540 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,540 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,542 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7397090748367343308] 11:36:54,543 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 11:36:54,547 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:54,547 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 11:36:54,555 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:54,555 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:54,560 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,561 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,564 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,568 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 11:36:54,568 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 11:36:54,571 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#162172964] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,577 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 11:36:54,577 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,577 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,580 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,581 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@389a5076} 11:36:54,581 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 11:36:54,581 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,583 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,584 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,584 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,585 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,586 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,586 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,587 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,587 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,587 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,588 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,588 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 11:36:54,588 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 11:36:54,588 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1208205052 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#26838239 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-2022887758] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#-751558664] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1868968689] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1208205052] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,589 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#26838239] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,589 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,590 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,591 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,592 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,593 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,594 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 11:36:54,594 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 11:36:54,595 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 11:36:54,596 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,596 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,596 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,596 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,596 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 11:36:54,596 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 11:36:54,596 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,596 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,596 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,596 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,596 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,596 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,596 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,597 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,597 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@9ee47a} 11:36:54,598 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 11:36:54,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,607 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,607 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,608 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,608 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#8651310 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,608 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,608 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#392622504] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,608 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#-1176158587] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,608 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#8651310] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,609 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-544511735 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,609 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-544511735] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,610 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,610 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,613 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,613 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,613 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,613 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,616 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 11:36:54,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 11:36:54,616 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,616 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,616 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,616 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,616 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,617 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,617 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,617 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,617 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,617 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 11:36:54,617 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 11:36:54,617 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,617 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,626 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:36:54,634 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:36:54,641 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-113951180208331443] 11:36:54,641 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,644 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:54,645 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:54,645 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 11:36:54,646 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 11:36:54,646 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 11:36:54,661 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,663 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,665 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,671 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1703910414] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,671 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 11:36:54,671 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 11:36:54,673 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 11:36:54,674 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,674 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,679 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,680 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4bbce64f} 11:36:54,681 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 11:36:54,681 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,683 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:36:54,685 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,685 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,685 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,686 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,687 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,687 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 11:36:54,687 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,687 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 11:36:54,687 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,688 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,688 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,688 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#429447905 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,689 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1125528454 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,689 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1013700244] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1198502802] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-462901022] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#429447905] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1125528454] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,691 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 11:36:54,692 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,692 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:36:54,692 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,693 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,694 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 11:36:54,695 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,696 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,696 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,696 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,696 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 11:36:54,697 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:54,698 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,699 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,699 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,699 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,699 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,699 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,699 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,699 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0a04751e00@4b44a9ac} 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-502538624] 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,700 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#945194824] 11:36:54,700 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,701 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,702 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,702 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 11:36:54,702 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 11:36:54,702 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 11:36:54,702 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 11:36:54,702 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#945194824] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-502538624] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1496169534 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 11:36:54,702 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,702 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1496169534] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,702 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#969192199 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,703 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#969192199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 11:36:54,703 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 11:36:54,703 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,703 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,703 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 11:36:54,704 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 11:36:54,704 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:36:54,705 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,706 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,706 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,706 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:54,706 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 11:36:54,706 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 11:36:54,706 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,706 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,706 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,706 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 11:36:54,707 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 11:36:54,707 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 11:36:54,707 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,707 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,707 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 11:36:54,707 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:54,707 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 11:36:54,707 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 11:36:54,858 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 11:36:54,859 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,859 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,859 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,859 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 11:36:54,859 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 11:36:54,877 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 11:36:54,877 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,877 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,877 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 11:36:54,877 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 11:36:54,917 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 11:36:54,917 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,917 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 11:36:54,918 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 11:36:54,918 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 11:36:54,919 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:36:54,919 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 11:36:54,919 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 11:36:54,919 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 11:36:54,919 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 11:36:55,208 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 11:36:55,209 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 11:36:55,308 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 11:36:55,328 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 11:36:55,505 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 11:36:55,505 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,507 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,507 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,507 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,507 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,568 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 11:36:55,570 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,570 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,578 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 11:36:55,580 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,581 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:55,581 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:55,581 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49124f9b 11:36:55,581 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 11:36:55,581 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 11:36:55,581 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 11:36:55,583 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c413373 11:36:55,583 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 11:36:55,583 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,583 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@35a42a7f 11:36:55,583 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,587 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 11:36:55,587 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,587 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,587 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,587 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,587 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,587 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,589 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:55,589 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:55,589 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 11:36:55,589 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 11:36:55,589 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e464 11:36:55,589 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 11:36:55,590 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ed1a9e3 11:36:55,590 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 11:36:55,591 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 11:36:55,591 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6391ef60 11:36:55,659 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:36:55,676 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 11:36:55,677 AM [cluster-test-shard-dispatcher-17] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,678 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,685 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:55,685 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:55,685 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 11:36:55,685 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 11:36:55,685 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c16a4ce 11:36:55,686 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 11:36:55,686 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 11:36:55,686 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5bdf7271 11:36:55,686 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 11:36:55,686 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 11:36:55,686 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 11:36:55,687 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c96ec0d 11:36:55,687 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 11:36:55,718 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 11:36:55,719 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,721 AM [cluster-test-shard-dispatcher-17] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 11:36:55,728 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:36:55,728 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 11:36:55,728 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 11:36:55,728 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 11:36:55,728 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b8d63f1 11:36:55,728 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 11:36:55,728 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 11:36:55,728 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 11:36:55,728 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:55,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@15f2a8f 11:36:55,729 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 11:36:55,729 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1be48b01 11:36:55,729 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 11:36:55,729 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 11:36:55,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 11:36:55,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-237656367] 11:36:55,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 11:36:55,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:55,730 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#22778456] 11:36:55,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#22778456] 11:36:55,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#162172964] 11:36:55,730 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:55,730 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-2022887758] 11:36:55,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-2022887758] 11:36:55,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1703910414] 11:36:55,730 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:55,730 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1013700244] 11:36:55,731 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1013700244] 11:36:55,731 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-2=false, member-3=false} 11:36:55,731 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 11:36:55,731 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:55,732 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,734 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]} has been successfully replicated to a majority of followers 11:36:55,736 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,736 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 11:36:55,736 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard ShardName{value=cars} 11:36:55,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-237656367] 11:36:55,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 11:36:55,739 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#22778456] 11:36:55,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#22778456] 11:36:55,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#162172964] 11:36:55,740 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 11:36:55,740 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-2022887758] 11:36:55,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-2022887758] 11:36:55,740 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 11:36:55,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1703910414] 11:36:55,741 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 11:36:55,741 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 11:36:55,741 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1013700244] 11:36:55,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-1013700244] 11:36:55,741 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:55,741 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,741 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,741 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 11:36:55,742 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,742 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,742 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 11:36:55,742 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 11:36:55,742 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 11:36:55,742 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 11:36:55,743 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 11:36:55,743 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,743 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,743 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 11:36:55,743 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,743 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,743 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 11:36:55,743 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 11:36:55,743 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 11:36:55,743 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 11:36:55,744 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:55,744 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:55,749 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:55,749 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:55,750 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:55,750 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:55,751 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:55,758 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:55,759 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:55,759 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 11:36:55,759 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 11:36:55,759 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,759 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,760 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 11:36:55,761 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 11:36:55,761 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 11:36:55,761 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 11:36:55,761 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 11:36:55,762 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:55,762 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:55,762 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 11:36:55,762 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,762 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:55,762 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 11:36:55,763 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 11:36:55,764 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 11:36:55,764 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 11:36:55,764 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:55,764 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:56,525 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 11:36:56,764 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-113951180208331443] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:36:56,781 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:56,781 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:36:56,783 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:36:56,785 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7397090748367343308] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [08/21/2025 11:36:56.785] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:36:56,790 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 11:36:56,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 11:36:56,790 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 11:36:56,790 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 11:36:56,791 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e061062 11:36:56,791 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c10c8dd 11:36:56,791 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e061062 11:36:56,791 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c10c8dd 11:36:56,791 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 11:36:56,791 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 11:36:56,791 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e43650d 11:36:56,791 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e72df7d 11:36:57,174 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 11:36:57,174 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 383.9 ms 11:36:57,175 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 11:36:57,175 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:57,176 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 11:36:57,324 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 11:36:57,324 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 533.6 ms 11:36:57,324 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 11:36:57,325 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:57,325 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 11:36:57,326 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 11:36:57,326 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 11:36:57,326 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 11:36:57,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 11:36:57,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 11:36:57,327 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 11:36:57,327 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e4f78c8 11:36:57,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e23e6a0 11:36:57,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e23e6a0 11:36:57,327 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 11:36:57,327 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d56a315 11:36:57,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35a41f91 11:36:57,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35a41f91 11:36:57,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 11:36:57,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 11:36:57,825 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 497.8 ms 11:36:57,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 11:36:57,826 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:57,826 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 11:36:57,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 11:36:57,924 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 596.9 ms 11:36:57,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 11:36:57,925 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 11:36:57,925 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 11:36:57,926 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 11:36:57,926 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 11:36:57,928 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:36:57,928 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:36:57,931 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/21/2025 11:36:57.933] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config